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Abstract 

This  study  investigated  the  data  collection  environment 
under  the  Core  Automated  Maintenance  System  (CAMS) ,  a 
computerized  information  system  used  in  the  Air  Force's 
aircraft  maintenance  complexes  to  aid  maintenance  managers  in 
maintaining  aircraft.  The  research  used  a  survey  to  measure 
maintenance  personnel's  perception  of  the  nature,  extent,  and 
causes  of  data  inaccuracies  occurring  in  the  CAMS  data 
collection  environment.  The  nature  was  differentiated  as 
being  either  intentional  or  accidental,  with  10  %  of  the 
total  errors  occurring  attributed  to  intentional  causes  while 
90%  were  attributed  to  accidental  causes.  Maintenance 
personnel  felt  that  difficulty  in  entering  information  into 
CAMS  accounted  for  the  largest  percentage  of  intentional 
errors  and  that  insufficient  training  on  CAMS  accounted  for 
the  largest  percentage  of  accidental  errors.  Additionally, 
the  results  of  this  research  were  compared  to  the  results  of 
earlier  research  conducted  by  Capt  Thomas  Folmar  who  measured 
the  perceptions  of  data  inaccuracy  under  the  Maintenance  Data 
Collection  (MDC)  system;  the  system  which  CAMS  replaced.  In 
general,  the  rank  of  the  maintenance  personnel  had  a 
significant  effect  on  the  way  they  perceived  data  inaccuracy. 
The  workers  (AB-SSgt)  perceived  no  increases  in  data  accuracy 
under  CAMS,  while  the  supervisors  (TSgt-CMSgt)  and  managers 
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(21t-Col)  were  mixed  in  their  perceptions  of  data  accuracy 
improvement  under  CAMS. 

The  maintenance  personnel  were  asked  what  they  perceived 
to  be  the  most  significant  contributor  to  data  inaccuracy  in 
the  CAMS  environment  and  what  actions  they  would  take  to 
improve  the  situation.  There  was  an  overwhelming  perception 
that  lack  of  training  on  CAMS  was  the  most  significant  cause 
of  data  errors  and  a  suggestion  that  a  training  program  may 
help  to  improve  the  data  accuracy. 

In  conclusion,  the  evidence  presented  in  this  research 
supports  characterizing  the  nature  of  errors  as  intentional 
or  accidental.  Additionally,  the  evidence  suggests  that  the 
research's  initial  attempt  to  categorize  the  causes  of  data 
errors  occurring  was  successful.  Finally,  rank  was  a 
significant  factor  in  differentiating  maintenance  personnel's 
perceptions  of  data  inaccuracy. 
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INACCURATE  DATA  ENTRY 
INTO  THE  AIR  FORCE 
MAINTENANCE  DATA  COLLECTION  SYSTEM 


I.  Introduction 


General  Issue 

Decision  makers  throughout  the  government  use 
computerized  information  systems  as  an  integral  tool  in  their 
decision  making  processes.  The  Core  Automated  Maintenance 
System  (CAMS)  is  a  computerized  information  system  "used  at 
base-level  to  manage  maintenance  equipment  and  personnel 
resources"  (AFM  66-279,  Vol  1:2-1).  Additionally,  CAMS 
"provides  much  of  the  maintenance  data  needed  by  major 
commands.  Air  Force  Logistics  Command,  Headquarters  USAF,  and 
other  agencies  to  manage  and  track  maintenance  resources 
worldwide"  (AMF  66-279,  Vol  1:2-1).  Implicit  in  providing 
maintenance  data  is  the  objective  of  supplying  maintenance 
decision  makers  with  information  that  is  accurate,  timely, 
and  in  a  format  that  is  easily  understood.  If  any  one  of 
these  characteristics  is  missing,  then  incorrect  decisions 
may  result.  Consequently,  it  is  valid  to  consider  whether 
any  of  these  characteristics  are  missing  from  the  maintenance 
information  collected  within  the  Air  Force's  maintenance 
complexes.  To  begin  to  address  this  issue,  this  research 
examined  the  accuracy  of  information  being  input  into  CAMS. 


1 


Problem  Statement 


Aircraft  maintenance  data  must  be  accurate  in  order  for 
it  to  be  useful  for  analysis.  Recently,  the  United  States 
Air  Force  implemented  CAMS  as  a  means,  among  other  things,  of 
collecting  maintenance  information.  It  is  not  clear  to  what 
extent  CAMS  has  affected  the  presence  of  data  inaccuracy 
previously  experienced  using  the  "papei — based"  Maintenance 
Data  Collection  (MDC)  System  which  CAMS  replaced 
(Folmar : 1986) .  Therefore,  there  is  a  continuing  need  to 
examine  the  accuracy  of  information  input  into  CAMS. 

Background 

This  research  is  a  follow-on  to  research  initiated  by 
Capt  Thomas  Folmar  who  investigated,  in  1986,  "Intentional 
Input  of  Errors  into  the  Maintenance  Data  Collection  (MDC) 
System"  (Folmar : 1986) .  His  research  focused  on  identifying 
"the  perceived  magnitude  of  intentional  errors  in  the 
Maintenance  Data  Collection  system  database  and  determining 
the  underlying  causes  for  the  reporting  of  inaccurate  and 
invalid  data"  (Folmar:4).  Although  Folmar  never  formally 
defined  an  "intentional  error,"  he  implied  a  definition  by 
stating  that  his  research  effort  "focused  on  the  possibility 
that  personnel  may  intentionally  input  inaccurate  and  invalid 
data"  (Folmar : 5) .  Additionally,  he  associated  inputting 
intentional  errors  with  the  integrity  of  personnel  inputting 
the  data  (Folmar : 16-17) .  Therefore  it  was  felt  that  Folmar's 
definition  of  an  "intentional  error"  was  consistent  with  the 
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following  definition  used  in  this  research.  An  intentional 
error  is  one  in  which  the  maintenance  technician  knows  the 
data  is  incorrect  at  the  time  of  entry. 

Folmar  suggested,  as  a  possible  cause  of  intentional 
errors,  that  some  maintenance  technicians  lacked  integrity, 
and  therefore  intentionally  falsified  data  input  into  the  MDC 
system  (Folmar : 17-19) .  However.  Folmar  also  reported  that 
"unintentional  errors"  were  being  input  into  the  MDC  system 
by  maintenance  technicians  (Folmar:17).  Additionally,  Folmar 
discovered  a  General  Accounting  Office  Report  condemning  the 
poor  quality  of  the  information  collected  in  the  MDC  system 
(Folmar : 14-16) .  The  Air  Force  response  to  the  GAO  report 
suggested  that  CAMS  would  solve  many  of  the  problems  of  the 
"paper-based"  MDC  system  (Folmar:16).  It  is  clear  that  CAMS 
automated  the  data  collection  process,  but,  as  with  many 
information  systems,  it  is  not  clear  how  CAMS  has  affected 
the  data  collection  environment. 

Justification 

The  fundamental  reasons  to  collect  maintenance 
information  have  remained  constant.  The  information  is  a 
resource  which  maintenance  managers  use  to  keep  the  Air  Force 
capable  of  performing  its  mission.  The  accuracy  of 
maintenance  information  is  affected  by  many  factors, 
especially  the  data  collection  environment.  By  examining  the 
data  collection  environment,  it  may  be  possible  to  draw  some 
important  conclusions  about  data  inaccuracy.  Folmar  examined 
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data  inaccuracy  under  the  "paper-based"  MDC  system,  whereas 
CANS  has  "automated"  the  data  collection  process.  By 
examining  the  differences,  if  any,  between  these  two  systems, 
this  research  provides  insights  to  the  changes  in  data 
inaccuracy  under  the  CAMS  environment.  Also,  by  identifying 
the  extent,  nature  and  causes  of  perceptions  of  data 
inaccuracies  in  the  CAMS  environment,  this  research  lays  the 
foundation  for  other  studies  to  examine  different  methods  to 
control  data  inaccuracy. 

Research  Objectives 

This  research  had  three  primary  objectives,  the  first  of 
which  concerned  the  nature,  causes,  and  extent  of  data 
inaccuracies  being  input  into  CAMS.  Additionally,  the 
research  examined  whether  perceptions  of  data  inaccuracies 
varied  across  Major  Commands  (MAJCOMS)  and  rank  structure. 
Finally  the  change  in  data  inaccuracy  under  the  CAMS 
environment  was  examined  by  comparing  selected  results  from 
the  first  two  objective  areas  of  this  research  to  the  results 
obtained  by  Folmar  (Folmar : 1986)  for  users  of  the  MDC  system. 

Investigative  Questions 

The  following  questions  were  answered  in  order  to 
acquire  the  desired  understanding  of  data  inaccuracy  with 
respect  to  the  CAMS  environment. 

1.  What  percentage  of  the  information  being  input  into 
CAMS  is  perceived  to  be  inaccurate? 
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2.  What  percentage  of  the  inaccuracies  being  input  into 
CAMS  are  intentional?  Accidental? 

3.  What  are  the  possible  causes  of  intentional  errors? 
Accidental  errors? 

4.  How  much  do  the  possible  causes  identified 
contribute  to  each  type  of  error? 

5.  Are'  the  inaccuracies  consistent  across  commands  and 
rank  structure? 

6.  Are  the  inaccuracies  and  perceived  information 
value,  identified  with  respect  to  CAMS, 
significantly  different  from  the  inaccuracies  and 
perceived  information  value  identified  by  Capt 
Thomas  Folmar  (Folmar : 1986) . 

The  first  investigative  question  examined  the  extent  of 
data  inaccuracy  and  investigative  question  2  identified  the 
nature  of  those  inaccuracies.  Investigative  questions  3  and 
4  provided  a  framework  to  identify  the  causes  of  data 
inaccuracy  and  determined  which  of  those  causes  are  most 
prevalent.  Investigative  question  5  provided  the  framework 
to  examine  numerous  factors  which  could  suggest  variations  in 
data  inaccuracy  with  respect  to  CAMS.  Investigative  question 
6  presented  a  basis  for  assessing  whether  the  data  collection 
environment  improved  after  CAMS  was  implemented. 

Methodology 

Capt  Folmar  used  a  survey  instrument  as  a  basis  for 
measuring  the  perspectives  of  MDC  users.  Therefore,  it  was 
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deemed  most  appropriate  to  use  a  survey  instrument  to 
facilitate  a  valid  comparison  to  Folmar's  study.  Also,  a 
survey  instrument  provided  the  greatest  coverage  of  the 
population  within  time  and  cost  constraints.  This  allowed 
the  results  to  be  more  "general izable"  than  other  methods. 
Additionally,  surveying  is  an  ideal  tool  to  measure  the 
perceptions  of  others.  Finally,  it  was  felt  that  a  survey 
provided  the  respondents  with  anonymity,  which  lessened  their 
fears  of  repercussions,  and  thus  increased  the  response  rate. 

The  survey  questions  included  those  used  by  Folmar  to 
measure  intentional  errors  (Folmar : 69-73) ,  but  also  expanded 
on  his  research  by  quantifying  the  nature,  causes,  and  extent" 
of  accidental  errors  as  well.  Additionally,  the  current 
research  used  a  two-way  Analysis  of  Variance  (ANOVA)  to 
examine  whether  the  responses  were  consistent  across  MAJCOMS 
and  rank.  Finally,  using  the  appropriate  results  from 
Folmar's  study  and  this  research,  a  comparison  of  the  present 
automated  maintenance  data  collection  environment  under  CAMS 
and  the  "papei — based"  data  collection  environment  measured  in 
Folmar's  research  was  made  using  a  three-way  ANOVA.  The 
ANOVA  results  served  as  a  basis  for  determining  the  changes 
in  data  inaccuracy  under  the  CAMS  environment. 

Scope  and  Limitations 

Folmar's  investigation  of  the  causes  of  data  inaccuracy 
focused  primarily  on  "intentional"  errors  as  a  function  of 
the  integrity  of  the  technicians  inputting  the  data,  but  also 
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identified  that  "unintentional"  or  "accidental"  errors  were 
occuring  as  well  (Folmar : 17) .  This  idea  suggested  the 
framework  for  this  review.  The  current  research  broadened 
the  perspective  of  Folmar ‘s  initial  inquiry  by  acknowledging 
that  "intentional"  errors  occur,  but  further  asks:  Is  the 
integrity  of  the  technician  the  primary  cause  of  this  type  of 
error,  or  are  there  other  factors  that  significantly 
contribute  to  the  problem?  Additionally,  this  research 
attempted  to  specifically  identify  the  causes  of  "accidental" 
errors  as  well. 

Since  the  investigation  methodology  relied  on  a  survey 
instrument,  this  research  measured  "perceptions"  of  the 
accuracy  of  maintenance  data.  There  is  always  the 
possibility  that  perceptions  of  reality  and  reality  are 
different.  To  confirm  this  difference  field  measurements 
that  compare  actual  maintenance  actions  performed  to  those 
reported  could  be  accomplished.  However,  such  measurements 
are  beyond  the  scope  and  capability  of  this  investigation. 

Additionally,  it  was  necessary  to  change  some  of  the 
survey  questions  that  Folmar  used.  The  acronym  "CAMS" 
replaced  the  acronym  "MDC"  in  the  survey  questions.  This  was 
considered  to  be  a  potential,  but  minor,  source  of  error  in 
the  research.  Also,  in  some  cases  Folmar  had  asked  "dual" 
questions.  For  example,  he  asked  "The  majority  of  inaccurate 
and  invalid  data  that  is  input  to  the  MDC  system  is  caused 
by"  (Folmar:70)?  It  was  felt  in  this  research  that  to  ask 
about  both  "inaccurate"  and  "invalid"  data  confused  the 
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respondent.  Therefore  only  one  aspect  of  the  dual  questions 
were  asked  at  a  time.  This  limited  the  comparisons  to 
Folmar's  research  and  confounded  the  inferences  that  could  be 
made.  These  issues  are  addressed  in  greater  detail  in 
Chapter  3. 

Finally,  this  research  is  not  meant  to  support  or 
condemn  current  maintenance  data  collection  practices,  but 
rather  to  shed  light  on  the  nature  of  the  inaccuracies 
occurring  and  further  the  understanding  of  the  environment 
which  causes  these  errors. 
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II.  Examination  of  Relevant  Literature 


Method  of  Treatment  and  Organization 

Data  entry  errors  probably  result  from  complicated  and 
interrelated  factors.  One  such  factor  is  the  technological 
or  "human  factors"  problem  of  transcribing  alpha  and  numeric 
codes  into  a  computer.  A  second  factor  is  the  motivation  of 
maintenance  personnel  to  correctly  accomplish  this 
"administrative"  part  of  their  job.  To  facilitate  this 
research  two  basic  areas  of  literature  were  examined.  The 
first  is  the  ergonomics  literature  that  pertains  to 
transcribing  alpha  and  numeric  characters  from  printed  lists 
to  keyboards  (and  similar  exercises).  Because  the  technology 
in  this  area  seems  to  be  mature,  the  literature  available 
seemed  to  be  old  (the  latest  article  identified  by  this 
research  was  written  in  1977)  and  therefore  will  be  presented 
in  a  historical  sequence.  The  second  area,  in  the  field  of 
"human  behavior."  pertains  to  motivation  and  will  be 
presented  by  analyzing  the  relevance  of  the  literature  to 
this  research. 

Literature  Discussion 

Ergonomics  Techno locrv:  Ideal  Arrangement  of 
Characters .  The  earliest  prominent  article  discovered  for 
this  review  is  by  John  L.  Coffey  of  the  Battel le  Memorial 
Institute.  Columbus  Ohio.  In  "  A  comparison  of  Vertical  and 
Horizontal  Arrangements  of  Alpha-Numeric  Material-Experiment 
1"  (Cof fey ; 93-98) .  Coffee  tries  to  determine  the  "relative 
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effectiveness  of  visual  displays  containing  alpha-numeric 
material  displayed  in  vertical  and  horizontal  arrangements" 
(Coffey:93).  He  tested  several  subjects'  abilities  to 
quickly  and  accurately  read  numbers  and  digits  from  strings 
of  digits  and  alpha  characters  arranged  in  various  sequences 
of  rows  or  columns.  This  experiment  measured  only  the 
subjects'  ability  to  read  information  but  not  to  transcribe 
the  information  that  they  read.  The  major  finding  of 
Coffey's  experiment  was 

the  non-significant  arrangement  of  the  arrangement 
variable.  It  was  found  that,  for  all  practical 
purposes,  the  different  effects  of  vertical  and 
horizontal  arrangement  of  the  alpha-numeric  materials 
on  operator  performance  are  negligible.  (Coffey; 98) 

Coffey's  results  were  interesting  and  led  others  into 

this  area  of  investigation.  C.M.  Williams,  of  Bell 

Laboratories,  conducted  a  similar  experiment  that  measured 

the  arrangement  of  horizontal  vs.  vertical  information 

display.  In  Williams'  experiment  "a  task  was  constructed  to 

compare  performance  on  a  horizontal  to  that  of  a  vertical 

array  of  3-digit  numbers"  (Wi 1 1 iams : 237) .  Again,  this  test 

measured  only  the  subjects'  ability  to  "locate  and  mark,  as 

quickly  and  as  accurately  as  they  could,  the  deviant  member 

within  each  set  of  four  pairs'^  (Wi  1 1  iams : 237)  .  The 

experiment  did  not  test  the  subjects'  ability  to  transcribe 

any  information.  Williams  found  that 

The  average  time  required  to  complete  the  vertical 
array  was  73  seconds  and  44  seconds  for  the  horizontal. 
The  finding  that  an  average  of  66%  more  time  was  spent 
on  the  vertical  than  on  the  horizontal  array  is 
significant  at  the  .005  level.  (Wi 1 1 iams : 237) 
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This  result  is  obviously  much  different  from  Coffey's 

results  and  Williams  explains  this  difference  by  stating 

Coffey  asked  direct  questions  about  a  single  letter, 
digit,  or  two-letter  word  within  the  array  that  could 
have  been  answered,  with  the  exception  of  counting,  by 
simply  scanning  the  array  until  the  item  was 
recognized.  The  subjects  were  never  required  to  read  a 
series  of  letters  or  numbers,  yet,  the  conclusion 
concerning  operator  performance  implies  inclusion  of 
reading  as  well  as  recognition  tasks.  The  point  to  be 
made  is  that  the  conclusion  drawn  exceeded  the  limits 
of  the  experiment.  (Williams: 238) 

In  1972,  Rodney  M.  Woodward  of  the  North  American 

Rockwell  Corporation,  Los  Angeles  Division,  decided  to 

investigate  the  differences  between  the  earlier  experiments 

that  examined  vertical  vs.  horizontal  arrangement  of 

information.  Woodward  conducted  an  experiment  that  "used  a 

visual  comparison  of  numeric  materials  to  examine  the  factors 

of  proximity  and  direction  of  arrangement,  and  attempts  to 

reconcile  the  differences  in  earlier  experiments" 

(Woodward; 337) .  This  experiment  required  subjects  to  quickly 

and  accurately  pick  3-digit  numbers  from  rows  or  columns  of 

numbers,  then  verbally  produce  the  results  (Woodward : 338) , 

The  experiment  did  not  require  the  subjects  to  transcribe  any 

information.  Woodward’s  experimental  results  "support  the 

data,  though  not  all  the  conclusions,  of  the  earlier  studies 

by  Williams  (1966)  and  Coffey  (1961)"  ( Woodward: 341 ) . 

Woodward's  final  conclusion  in  this  experiment  is  that  there 

is  a  difference  in  whether  information  is  displayed 

horizontally  or  vertically,  but  it  is  a  function  of  the 

interaction  of  proximity  and  arrangement  (Woodward : 338 ) . 
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Ergonomics  Technology:  Experiments  in  Transcribing 

Data .  The  next  series  of  experiments  reviewed  concentrates 

on  copying  information  from  one  list  to  another.  The  first 

study  reviewed,  was  an  experiment  accomplished  in  1967  by 

William  J.  Smith,  Jr.  titled  "Accuracy  of  Manual  Entries  in 

Data-Col lection  Devices"  (Smith: 1967) .  In  this  experiment. 

Smith  examined  the  "accuracy  of  manually  recorded  messages 

similar  to  those  encountered  in  field  studies  on  the  accuracy 

of  data  collection  in  production  information  systems" 

(Smith:362).  He  successfully  simulated  some  of  the  field 

environment  in  the  laboratory  to  examine  the  nature  of  data 

entry  errors.  He  classified  the  types  of  errors  he  wanted  to 

identify  into  three  major  categories:  message  format, 

message  content,  and  event  description.  During  the  course  of 

his  experiment  Smith  noted  that 

Failure  to  record  events,  or  omitted  entries,  proved  to 
be  a  special  concern  in  both  the  accuracy  of  the  system 
and  the  design  of  processing  techniques.  Such 
inaccuracies  are  difficult  to  measure  in  field  studies 
and  awkward  to  emulate  in  laboratory  tasks.  (Smith: 363) 

Smith's  observation  has  particular  relevance  to  this 

research.  If  these  types  of  errors  are  difficult  to  measure 

in  field  studies  and  difficult  to  emulate  in  the  laboratory. 

then  their  detection  in  the  "real  world"  will  also  be 

difficult  and  their  impact  difficult  to  assess.  In  fact. 

Smith  further  notes  that  the  only  way  to  detect  these  types 

of  errors  is  through  "post-processing  analysis"  (Smith:363). 

It  should  be  noted  that  this  research  did  not  attempt  to 

identify  or  quantify  this  type  of  error. 
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In  1977  John  N.T.  Martin,  John  Morton,  and  Pennie  Ottley 

performed  an  experiment  titled  "Experiments  on  Copying  Digit 

Strings."  In  this  experiment  they  state 

A  combination  of  increasing  use  of  numerical  referents 
for  goods  and  increasing  computerization  of  transactions 
means  that  more  people  have  jobs  which  involve  copying 
lists  of  relatively  meaningless  symbols  such  as  numeric 
codes.  When  asked  to  give  advice  about  such  situations, 
either  on  the  design  of  codes  or  the  layout  of  the 
machinery,  we  keep  finding  that  much  of  the  necessary 
data  is  missing  from  the  literature.  (Martin  et  al:409) 

The  purpose  of  their  experiment  was  to  determine  some  of 

the  parameters  that  would  help  them  to  answer  the  designer's 

questions.  They  examined  such  factors  as  where  to  locate  the 

input  data  (above  or  to  the  side  of  a  keyboard),  whether  the 

data  should  be  in  column  or  row  format,  and  whether  using 

separators  (hyphens  or  spaces)  in  long  digit  strings  affected 

the  transcription.  Their  results  and  conclusions  indicate 

that  there  was  a  preference  for  locating  the  source  of  the 

input  data  above  the  keyboard  and  that  there  was  a  definite 

advantage  to  using  horizontal  as  opposed  to  vertical 

arrangement  of  the  digit  strings  (Martin  et  al:416). 

Summary  of  Ergonomics  Studies.  The  earlier  research  in 

this  area  concentrated  mainly  on  recognition  of  information 
in  vertically  or  horizontally  arranged  alpha-numeric 
sequences.  This  task  was  combined  in  later  studies  to 
include  transcribing  information  into  computer  systems.  The 
research  indicates  that  there  is  preference  for  information 
arranged  horizontally,  with  the  source  of  this  information 
located  above  the  keyboard  (Martin  et  al:416).  These  studies 
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are  relevant  to  this  research  because  the  maintenance 
technician  is  required  to  transcribe  numeric  codes  from 
technical  manuals  into  CAMS. 

Human  Motivation.  The  second  area  of  examination  of 
this  research  is  aimed  at  understanding  how  to  motivate 
maintenance  personnel  to  do  their  job  properly,  ie.  correctly 
completing  the  "administrative"  functions  of  inputting  the 
proper  information  into  CAMS.  Indeed,  motivating  people  to 
do  what  they  should  do  has  always  been  a  dilemma  to  managers. 
In  the  maintenance  complex,  motivating  maintenance  personnel 
to  perform  their  jobs  can  be  quite  difficult.  Aircraft 
maintenance  is  a  high  pressure  job  that  demands  much  of 
people.  The  "push"  to  meet  flying  and  training  schedules  in 
the  Air  Force  environment  focuses  most  of  management's 
attention  on  "operational"  matters,  often  to  the  neglect  of 
"administrative"  functions.  Additionally,  there  are  no 
standard  procedures  for  data  entry  in  the  CAMS  environment. 
Ideally,  each  technician  should  personally  enter  the 
information  immediately  after  performing  every  maintenance 
action.  However,  this  seldom  happens  because  time  and 
manning  constraints  are  frequently  severe.  The  maintenance 
technician  may  have  to  wait  until  the  end  of  the  day  to  enter 
information  or  may  have  to  give  the  information  to  a  single 
person  performing  data  entry  for  everyone.  Obviously,  the 
technicians  have  many  demands  with  which  to  contend  in 
addition  to  entering  data  on  the  maintenance  actions  they 
performed  during  the  course  of  the  day.  Because  these 
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demands  and  constraints  are  present  in  the  maintenance 
technician's  world,  they  have  a  direct  bearing  on  the  data 
entry  environment.  To  provide  a  better  understanding  of  this 
problem,  current  literature  on  motivational  theory  was 
examined . 

There  are  several  theories  on  what  motivates  people. 

One  could  begin  by  looking  at  Maslow’s  hierarchy  of  needs 

(Matteson  and  Ivancevich : 369-389)  or  Herzberg ‘ s  motivators 

and  hygiene  factors  (Daft  and  Steers : 168-169)  to  explain 

human  motivation.  However,  there  are  several  reasons  these 

theories  do  not  provide  a  good  framework  for  this  research. 

First,  it  can  be  difficult  to  locate  some  individuals  on 

Maslow's  hierarchy.  Applying  Maslow's  hierarchy  to  attempt 

to  find  why  maintenance  technicians  do  not  enter  data 

correctly,  if  at  all,  into  CAMS  would  prove  quite  cumbersome. 

In  addition,  some  aspects  of  Herzberg 's  theory  concerning 

motivators  and  hygiene  factors  are  inappropriate  to  explain 

why  maintenance  personnel  do  not  enter  accurate  data.  For 

example,  examining  how  individual  promotions  are  directly 

affected  by  maintenance  personnel  entering  correct  data  into 

CAMS  is  meaningless.  Therefore,  a  framework  that  examines 

motivation  for  specific  behavior  is  necessary  to  examine  the 

CAMS  data  collection  environment.  This  framework  is  provided 

by  Expectancy  Theory  which  states  that  human  behavior 

is  a  function  of  the  value  of  the  reward  the  doer 
perceives  as  coming  from  the  chosen  behavior  and  the 
doer's  expectation  that  the  reward  is  attainable  without 
undue  risk  or  effort.  (Quick: 96) 
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There  are  several  recent  studies  that  relate  motivation 


to  perceived  reward  and  required  effort  to  obtain  rewards. 

For  example,  James  R.  Bettman.  Eric  J.  Johnson  and  John  W, 
Payne  state.  "A  major  finding  of  the  last  decade  of  decision 
research  is  that  an  individual  may  use  many  different  kinds 
of  strategies  in  making  a  decision  contingent  upon  task 
demands"  (Bettman  et  alrlll).  This  statement  clearly 
delineates  the  connection  between  task  effort  and  strategies 
for  goal  attainment.  Additionally,  they  state  in  their 
conclusions  that  "the  concept  of  effort  plays  a  major  role  in 
attempts  to  understand  the  contingent  use  of  processing 
strategies"  (Bettman  et  al:134). 

Robert  J.  Graham  noted  a  relationship  between  data 
accuracy  and  personal  motivation  to  enter  accurate  data.  He 
makes  the  following  observations  about  computers,  their 
operators,  and  some  faulty  coiranon  assumptions. 

1.  Data  reflects  a  constant  reality. 

2.  People  are  behaving  according  to  the  rules. 

3.  People  will  do  what  they  say  for  the  reasons  they 
say. 

4.  Production  of  data  is  not  affected  by  organizational 
politics.  ( Graham : 40-43 ) . 

The  main  point  Graham  wishes  to  make  by  stating  the 
above  observations  is 

People  have  motives  and  purposes  and  so  when  they  have 
the  ability  to  influence  data  they  will  most  likely 
influence  it  to  suit  their  purposes.  Since  all  data  are 
produced  by  people,  we  can  assume  that  all  data  are 
biased.  (Graham:43) 

Any  researcher  or  person  who  desires  to  understand  the 
interaction  between  the  computer  system  and  its  human 
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operators  must  understand  the  bias  and  motives  of  the  human 
operators  (Graham: 43) . 

Conclusion 

This  research  identified  a  body  of  literature  that 
provides  some  understanding  of  the  dual  problem  of  technology 
and  human  motivation  as  it  relates  to  data  entry  into  CAMS, 
the  United  States  Air  Force's  maintenance  data  collection 
system.  Several  experiments  and  articles  relating  to  the 
human  factors  aspect  of  reading  and  entering  data  strings 
into  computer  systems  were  identified  and  presented. 
Additionally,  some  .articles  were  identified  that  addressed 
human  motivation  to  provide  a  framework  for  understanding  why 
humans  make  some  types  of  decisions.  An  area  of  literature 
that  this  research  was  not  able  to  identify  was  literature 
that  specifically  studied  all  aspects  of  the  maintenance  data 
collection  environment  at  one  time.  Specifically,  in  the 
body  of  literature  that  was  reviewed,  no  experiments  were 
conducted  that  evaluated  the  interaction  between  the 
motivation  of  operators  to  enter  accurate  information  and  the 
human  factors  elements  of  the  environment.  The  Graham 
article  "touched"  on  this  aspect  but  did  not  present  any 
empirical  evidence  to  support  his  position  that  all  data  are 
biased  because  of  the  motives  of  the  human  operators  that 
input  the  data.  Therefore,  further  investigation  into  the 
interaction  of  the  motivation  of  the  maintenance  technician 
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and  the  human  factors  element  of  the  CAMS  environment  is 
warranted . 
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IIL. _ Methodolocrv 


Introduction 

The  purpose  of  this  chapter  Is  to  describe  the  actions 
that  were  taken  to  conduct  the  research  described  in  chapter 
one.  It  explains  the  reasoning  behind  the  choice  of  methods 
and  their  implementation. 

Justification 

To  answer  the  investigative  questions,  a  survey 
instrument  was  developed.  There  were  several  reasons  for 
this  choice.  First,  the  survey  instrument  provided  the 
greatest  coverage  of  the  population  within  time  and  cost 
constraints.  This  allowed  the  results  to  be  more 
"general izable"  than  other  methods.  Also,  Folmar  (1986)  used 
the  survey  instrument,  so  it  was  necessary  to  replicate  this 
approach  to  facilitate  valid  comparisons  to  his  results. 
Additionally,  surveying  is  an  ideal  tool  to  measure  the 
perceptions  of  others.  For  example.  Emory  states  "One  can 
seldom  learn  much  about  opinions  and  attitudes  except  by 
questioning"  (Emory:158).  Finally,  it  was  felt  that  the 
survey  provided  respondents  with  anonymity,  which  lessened 
their  fears  of  repercussions,  and  thus  increased  both  the 
response  rate  and  the  frankness  of  responses  to  individual 
survey  items. 

Experimental  Design 

This  research  uses  a  "separate  sample  pretest-post-test 
design"  of  the  form: 
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R  Oj  (X) 

R  X  O, 

(Emory:127).  where  the  first  observation,  0^,  is  Folmar's 
study,  the  treatment,  X,  is  the  CAMS  implementation  and  the 
final  observation.  0,  .  is  this  research.  "The  bracketed 
treatment  (X)  is  irrelevant  to  the  purpose  of  the  study  but 
is  shown  to  indicate  that  the  experimenter  cannot  control  the 
treatment"  (Emory:127).  The  samples  drawn  for  Folmar's  study 
and  this  research  were  each  a  stratified  random  sample  of 
maintenance  technicians  and  managers  extracted  from  the  Air 
Force  Manpower  and  Personnel  Center's  (AFMPC)  ATLAS  computer 
system.  Emory  states  that  the  separate  sample  pretest-post¬ 
test  design  is  not  a  strong  design  because  a  number  of 
threats  to  internal  validity  are  not  handled  adequately 
(Emory: 127).  Indeed,  in  any  study  threats  to  internal 
validity  are  a  concern,  but  it  was  of  particular  interest  in 
this  research  because  the  experimenters,  Folmar  and  this 
author,  were  not  able  to  control  the  CAMS  implementation.  In 
this  research,  there  were  primarily  four  areas  of  concern  as 
threats  to  internal  validity  (Emory; 116)  as  a  result  of  this 
experimental  design: 

History.  There  may  be  some  historical  effects,  that  is, 
some  event  may  have  occurred  between  the  two  samples  that 
confused  data  inaccuracy.  For  example,  the  Air  Force's  shift 
to  "Rivet  Workforce",  a  workforce  compression  plan  to  use 
maintenance  personnel  more  efficiently,  may  have  caused  a 
shift  in  the  perceptions  of  data  inaccuracy  that  are  not 


20 


directly  related  to  the  CAMS  implementation.  However,  it  was 
felt  that  historical  factors  in  this  research  would  be 
difficult,  if  not  impossible,  to  control  and  presented  only  a 
minor  threat  to  internal  validity. 

Maturation.  Maturation  refers  to  "changes  that  take 
place  within  the  subject  which  are  a  function  of  the  passage 
of  time  and  are  not  specific  to  any  particular  event" 

(Emory: 116).  In  this  research  it  was  possible  that  the 
technicians  became  progressively  more  computer  literate  with 
the  passage  of  time.  This  may  have  affected  the  technicians 
perceptions  of  data  inaccuracy  with  respect  to  CAMS.  Again, 
this  aspect  of  internal  validity  is  difficult  to  control,  and 
was  not  considered  to  be  a  significant  threat  to  internal 
validity. 

Instrumentation .  "This  is  a  threat  to  internal  validity 
that  results  from  changes  between  observations,  in  measuring 
instrument  or  observer"  (Emory:116).  This  threat  was  present 
in  this  research.  However,  steps  were  taken  to  minimize  the 
possible  impact  on  this  research  when  the  survey  questions 
were  developed.  These  steps  are  described  in  detail  in  the 
section  "Applying  the  Survey  Instrument." 

Selection.  Emory  states  "One  of  the  more  important 
threats  to  internal  validity  is  the  differential  selection  of 
subjects  to  be  included  in  experimental  and  control  groups" 
(Emory: 116).  This  threat  was  addressed  by  using  random 
selection  in  both  Folmar's  study  and  this  research. 
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The  strongest  attribute  of  the  separate  sample  pretest- 
post-test  design  is  that  "this  design  is  considered  to  be 
superior  to  true  experiments  in  external  validity.  This 
strength  results  from  it  being  a  field  experiment  in  which 
the  samples  are  usually  drawn  from  the  population  to  which  we 
wish  to  generalize  our  results"  (Emory:127).  Generalizing 
the  results  of  this  research  to  the  population  of  maintenance 
technicians  and  managers  in  the  United  States  Air  Force  in 
the  CONUS  was  considered  an  essential  element  of  this 
research. 

Applying  the  Survey  Instrument 

Development  of  Survey  Questions.  Most  of  the  survey 
questions  used  in  this  research  are  adaptations  of  questions 
initially  posed  to  maintenance  technicians  by  Capt  Folmar.  A 
copy  of  the  survey  used  in  the  current  research  is  included 
as  Appendix  A.  The  identical  questions  could  not  be  used,  as 
would  normally  be  desirable  for  statistical  comparison, 
because  CAMS  replaced  the  "paper-based"  Maintenance  Data 
Collection  (MDC)  system  used  at  the  time  of  Folmar 's 
research.  However.  CAMS  uses  the  same  information  elements 
as  the  Air  Force  Technical  Order  (AFTO)  Form  349.  the  primary 
paper  form  used  for  maintenance  data  collection  at  the  time 
of  Folmar 's  research.  Therefore,  it  was  desirable  to  assume 
that  the  terms  MDC  and  CAMS  are  equivalent  when  used  in 
context  of  the  survey  questions.  So.  to  validate  the 
CAMS/MDC  equivalence  assumption,  several  maintenance 
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maintenance  technicians  and  maintenance  officers  who  have 
experience  with  both  systems  were  surveyed  to  evaluate  their 
perceptions  of  CAMS/MDC  equivalence  during  a  survey 
validation  pretest. 

Survey  Validation  and  CAMS/MDC  Equivalence  Results.  Six 
maintenance  personnel:  three  captains,  one  master  sergeant, 
one  technical  sergeant,  and  one  staff  sergeant,  were  used  to 
aid  in  validating  the  survey  used  in  this  research.  This  is 
a  small  number  but  they  were  the  only  personnel  locally 
available,  with  CAMS  experience,  to  validate  the  survey. 

This  small  number  was  considered  to  be  offset  by  the 
experience  level  of  the  personnel.  Two  of  the  captains  were 
the  CAMS  implementation  project  officers  in  their  respective 
organizations,  and  the  other  four  personnel  had  12  years 
combined  experience  using  the  CAMS  system.  Also,  2  of  the 
officers  were  fellow  members  of  the  Graduate  of  Logistics 
Management  (GLM)  Class  of  91-S,  AFIT,  and  were  therefore 
cognizant  of  the  concepts  of  validity  under  consideration. 

All  of  the  above  personnel  had  experience  with  both  the  CAMS 
and  MDC  systems.  Additionally,  they  were  told  that  the 
survey  used  in  this  research  was  developed  from  Folmar's 
survey  asking  the  same  or  similar  questions  about  the  MDC 
system.  After  completing  the  survey  used  in  this  research, 
they  were  asked  "Do  you  feel  the  terms  CAMS  and  MDC  are 
equivalent  in  the  context  of  the  survey  questions?"  all  six 
of  the  personnel  responded  that  they  felt  the  terms  were 
equivalent , 
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To  support  content  and  construct  validity,  the  pretest 
group  was  asked  to  provide  written  feedback  on  each  of  the 
questions  if  they  did  not  understand  the  question  or  had 
suggestions  that  they  thought  would  improve  the  quality  of 
the  questionnaire.  There  were  several  important  points  that 
the  personnel  contributed,  and  for  the  most  part,  the 
recommendations  were  incorporated  into  the  survey. 

External  validity  was  addressed  by  the  use  of  a 
stratified  random  sample.  Emory  states  “Stratification  is 
also  useful  when  the  researcher  wants  to  study  the 
characteristics  of  certain  population  subgroups"  (Emory:307), 
By  measuring  the  characteristics  of  the  various  strata 
indicated  in  this  study,  the  results  are  more  universally 
applicable  to  the  population  of  aircraft  maintenance 
technicians  and  managers  as  a  whole. 

Survey  Implementation.  To  provide  an  understanding  of 

the  "degree"  of  data  inaccuracy,  the  survey  questions  used 
interval  scales,  where  possible,  to  measure  the  respondents’ 
opinions.  This  choice  was  made  because,  as  Emory  states. 

"The  interval  scale  has  the  powers  of  nominal  and  ordinal 
scales  plus  one  additional  strength:  It  incorporates  the 
concept  of  equality  of  interval  (the  distance  between  1  and  2 
equals  the  distance  between  3  and  4)"  (Emory:91).  This 
concept  of  equality  was  necessary  for  the  ANOVA  used  in  this 
research. 

To  specifically  examine  how  this  research  used  the 
survey  instrument  to  explore  the  causes  of  data  inaccuracy. 
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each  investigative  question  will  be  stated,  followed  by  an 
explanation  of  survey  question  development  and  the 
statistical  analysis  used  to  interpret  the  survey  results. 

Investigative  Question  1.  What  percentage  of  the 
information  input  into  CAMS  is  inaccurate? 

There  are  two  main  constructs  in  this  investigative 
question.  The  first  is  "what  is  information?"  and  the  second 
is  "what  is  inaccurate?"  In  the  context  of  this  research, 
the  idea  of  information  was  considered  to  be  measured  in  two 
dimensions.  The  first  dimension  consists  of  the  individual 
data  elements  that  could  be  input  incorrectly,  and  the  second 
is  the  number  of  transactions  that  could  be  input 
incorrectly.  A  transaction  in  CAMS  terminology  is  the 
transmittal  and  storage  of  a  "screen"  (collection  of  data 
elements)  of  information  used  by  CAMS.  This  research 
quantified  the  percentage  of  inaccuracies  of  information 
occurring  only  at  the  data  element  level.  This  dimension  of 
the  construct  of  information  was  used  because  maintenance 
personnel  are  most  familiar  with  the  individual  data  elements 
that  are  input  into  CAMS.  The  idea  of  "inaccurate"  included 
any  reason  the  information  could  be  in  error.  These  reasons 
include  maintenance  technicians  entering  the  wrong  codes 
because  of  keystroke  errors,  lack  of  time,  lack  of 
motivation,  difficulty  in  using  the  T.O.  system,  or  honest 
mistakes  that  the  CAMS  edit  system  did  not  identify.  To 
ensure  the  maintenance  personnel  understood  that  the  survey 
was  measuring  data  inaccuracies  at  the  data  element  level,  an 
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Instruction  was  provided  that  explained  what  a  data  element 
was  in  relation  to  CAMS. 

The  percentage  of  data  input  into  CAMS  that  is 
inaccurate  was  measured  in  question  13.  The  respondents  were 
given  choices  on  a  scale  of  0^5-100%.  in  10  degree  increments, 
to  indicate  their  opinions.  A  histogram  was  used  to  show  the 
response  distribution.  The  mean,  mode,  and  standard 
deviation  were  calculated  for  each  of  the  demographic  groups 
that  participated  in  the  research.  The  mode  of  this 
distribution  was  the  main  statistic  examined  here.  This 
identified  the  percentage  of  information  input  into  CAMS,  at 
the  data  element  level,  that  the  greatest  number  of 
maintenance  personnel  felt  was  inaccurate. 

Investigative  Question  2.  What  percentage  of  the 
inaccuracies  being  input  to  CAMS  are  intentional? 

Accidental?  This  question  further  examines  the  nature  of  the 
data  inaccuracies.  Again,  there  are  two  constructs  of 
interest;  the  idea  of  "intentional"  and  "accidental"  errors. 
The  definition  of  an  intentional  error  is  an  error  that  is 
input  into  CAMS  by  a  maintenance  technician  with  full 
knowledge  that  they  have  input  erroneous  data.  Conversely, 
an  accidental  error  is  one  that  the  technician  does  not  know 
is  wrong  at  the  time  of  data  entry.  These  constructs  are 
best  illustrated  by  examples.  Maintenance  technicians  who 
input  codes  that  they  know  will  be  accepted  by  the  system, 
but  do  not  accurately  reflect  the  tasks  they  performed  are 
guilty  of  intentional  errors.  A  maintenance  technician  who 
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unknowingly  uses  a  wrong  code  because  the  T.O.  system  does 
not  clearly  indicate  which  code  should  be  used,  demonstrates 
an  accidental  error.  To  ensure  the  survey  respondents 
understood  the  definitions  of  intentional  and  accidental 
errors,  the  definitions  were  stated  in  the  questions 
measuring  these  items. 

Survey  questions  14  and  15  were  used  to  measure  the 
percentage  of  intentional  and  accidental  errors  respectively. 
A  histogram  was  used  to  show  the  response  distribution.  The 
mean.  mode,  and  standard  deviation  were  calculated  for  each 
of  the  demographic  groups  that  participated  in  the  research. 
The  mode  of  this  distribution  was  the  main  statistic  examined 
here.  The  answers  to  these  questions  identified  the 
percentage  of  the  total  errors  occurring  in  the  CAMS  system 
that  the  greatest  number  of  maintenance  personnel  felt  were 
attributable  to  intentional  and  accidental  errors 

Investigative  Question  3  and  4.  What  are  the 
possible  causes  of  intentional  errors  and  accidental  errors? 
With  respect  to  CAMS,  how  much  do  the  possible  causes 
identified  contribute  to  each  type  of  error?  There  are  two 
directly  related  constructs  in  these  two  questions.  The 
first  is  the  idea  of  "causes"  of  intentional  and  accidental 
errors  and  the  second  concerns  the  significance  of  each  of 
the  causes.  This  research  considered  it  impractical,  due  to 
the  time  required  to  analyze  the  results,  to  solicit  open- 
ended  opinions  of  the  causes  of  errors  in  CAMS.  Therefore, 
specific  survey  question  alternatives  were  developed  using 
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two  sources.  The  first  source  was  Folmar's  research,  which 
identified  some  probable  causes  of  both  intentional  and 
accidental  errors  (Folmar :46 ,48) .  Additionally,  the 
literature  on  expectancy  theory  (Quick: 96)  provided  some 
probable  causes  that  were  relevant.  In  addition,  the 
respondents  were  given  the  option  to  indicate  other  causes  in 
case  the  listed  causes  did  not  capture  the  full  scope  of 
relevant  possibilities.  The  validity  of  this  process  was 
supported  by  feedback  from  the  pretest  of  the  survey 
presented  earlier  in  this  chapter. 

Survey  questions  16-27  measured  percentages  of 
intentional  and  accidental  errors  attributed  to  causes 
indicated  in  the  survey.  A  histogram  was  used  to  show  the 
response  distribution  for  each  of  the  causes  presented.  The 
mean,  mode,  and  standard  deviation  were  calculated  for  each 
of  the  causes,  and  the  mean  was  the  main  statistic  of 
interest.  This  indicated  which  of  the  causes  presented 
accounts  for  the  largest  average  percentage  of  intentional 
and  accidental  errors. 

Investigative  Questions  5.  Are  the  inaccuracies 

consistent  across  MAJCOMS  and  rank  structure?  This  question 
examined  the  differences  that  could  occur  across  the 
different  demographic  groups  that  constitute  the  population. 
There  are  several  reasons  the  identified  groups  were  chosen. 
First,  Folmar  used  these  particular  groups  and  it  was 
desirable  to  have  the  same  groups  used  for  comparisons  in 
Investigative  Question  6.  Also,  the  differences  in  the 
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strategic  Air  Command's  and  Tactical  Air  Command's  operating 
procedures  may  have  had  an  impact  on  data  inaccuracy. 
Similarly,  the  rank  structures  could  have  had  a  significant 
impact  on  data  inaccuracy.  The  responses  to  survey  questions 
5-27  were  examined  using  a  two-way  ANOVA  to  answer  this 
investigative  question. 

Investigative  Question  6.  Research  Hypothesis: 

The  data  input  inaccuracies  and  perceived  information  value, 
identified  with  respect  to  CAMS,  are  respectively 
significantly  less  and  greater  than  the  inaccuracies  and 
perceived  information  value  previously  identified  by  Capt 
Thomas  Folmar.  This  hypothesis  examines  how  the  data 
collection  environment  has  changed  since  the  CAMS 
implementation.  It  is  stated  to  test  the  anticipated 
improvement  in  data  inaccuracy  and  perceived  information 
value  under  the  CAMS  data  collection  environment.  It  was 
expected  that  the  inaccuracies  identified  under  the  CAMS 
environment  would  be  less  than  those  Folmar  (Folmar : 1906) 
identified  and  the  perceived  information  value  would  have 
improved . 

Investigative  question  6  was  evaluated  using  a  three-way 
ANOVA.  The  variables  Major  Command  (MAJCOM) ,  rank,  and  study 
were  the  factors  used  in  the  ANOVA.  TAC  and  SAC  were  the 
levels  used  within  the  MAJCOM  factor.  Airman  Basic  (AB) 
through  Staff  Sergeant  (SSgt)  (subgroup  1:  "workers"). 
Technical  Sergeant  (TSgt)  through  Chief  Master  Sergeant 
(CMSgt)  (subgroup  2:  "supervisors"),  and  Second  Lieutenant 
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(21t)  through  Colonel  (Col)  (subgroup  3:  "managers")  were  the 
levels  used  within  the  rank  factor.  Finally,  this  research 
and  Folmar's  research  were  the  levels  used  within  the  study 
factor.  The  first  two  treatments  and  their  appropriate 
levels  were  chosen  because  Folmar  used  them  (Folmar:29)  and 
they  are  appropriate  ways  to  group  the  population  being 
studied  for  the  reasons  cited  above  under  investigative 
question  5.  The  three-way  ANOVA  compared  the  means  of  the 
responses  to  questions  5-11  and  14  for  this  study  to  the  same 
questions  in  Folmar's  study.  The  comparison  tested  for  any 
statistically  significant  difference  between  the  means  of  the 
responses  in  the  two  studies  while  still  controlling  for  the 
first  two  factors  of  MAJCOM  and  rank.  Bonferroni's  method 
(McClave  and  Benson:865)  was  used  to  isolate  specifically 
which  means  differed  significantly. 

Sample/Populat ion 

Total  Population.  The  population  studied  was  all  TAC 
and  SAC  military  aircraft  maintenance  personnel  in  the  ranks 
of  airman  basic  to  colonel.  At  the  time  of  the  research, 
there  were  a  total  of  61,270  personnel  in  this  population 
with  45,073  in  the  stratum  of  AB-SSgt  (workers),  14,611  in 
the  stratum  of  TSgt-CMSgt  (supervisors),  and  1566  in  the 
stratum  of  21t-Col  (managers) .  The  total  population  is 
presented  in  Table  I.  The  reason  Folmar  chose  a  stratified 
random  sample  was  that  "Stratification  is  almost  always  more 
efficient  statistically  than  simple  random  sampling  and  at 
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worst  is  equal  to  it:  With  the  ideal  stratification,  each 
stratum  is  homogeneous  internally  and  heterogeneous  with 
other  strata"  (Emory:307). 

Sample  Description.  The  driving  factor  of  sample  size 
in  this  research  was  the  restriction  that  a  sample  size  only 
large  enough  to  achieve  a  .10  level  of  significance  could  be 
drawn  from  the  ATLAS  database.  This  is  the  sample  size 
restriction  that  is  imposed  on  all  surveys  distributed  to  Air 
Force  personnel  by  the  Air  Force  Manpower  and  Personnel 


TABLE  1 

TOTAL  POPULATION. 


SAC 

TAC 

AB-SSGT 

16,550 

28,523 

TSGT-CMSGT 

5,645 

8,966 

2LT-C0L 

700 

886 

TOTALS 

22,895 

38,375 

Center  (AFMPC) .  The  following  formula  was  used  to  compute 
the  sample  sizes  shown  in  Table  2  {Folmar:32). 

(Jtfyz»x0.25)  (1) 

[d2x(N-l)]  +<Z2x0.25) 


where 

n  -  sample  size  required 

N  -  total  population  size  (known  or  estimated) 
d  -  level  of  significance 

Z  -  Z  factor  for  each  level  of  significance  (1.65  for 
.10  level  of  significance) 
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The  resulting  sample  sizes  are  shown  in  Table  2, 

TABLE  2 
SAMPLE  SIZE 


SAC 

TAC 

AB-SSGT 

75 

75 

TSGT-CMSGT 

60 

60 

2LT-C0L 

60 

60 

TOTALS 

195 

195 

The  figures  in  Table  2  were  multiplied  by  a  factor  of  2.5  for 
the  stratum  of  AB-SSgt  and  2.0  for  the  remaining  strata  to 
account  for  survey  non-return. 

Data  Collection  Plan 

General .  The  names  of  the  subjects  to  be  surveyed  were 
obtained  from  the  ATLAS  database.  To  limit  the  number  of 
names  extracted  from  the  database  and  to  achieve  the 
randomization  desired  for  this  research,  the  numbers  0-9  were 
placed  in  an  envelope  and  drawn  1  at  a  time  at  random.  The 
numbers  drawn  were  used  to  indicate  the  last  digit  of  the 
social  security  number  of  maintenance  personnel  to  extract 
from  the  ATLAS  database.  Additionally,  the  search  was 
limited  to  personnel  stationed  at  Continental  United  States 
(CONUS)  TAC  and  SAC  Air  Force  bases.  The  surveys  were  mailed 
to  the  personnel  extracted  during  the  data  base  search  and 
the  respondents  were  given  two  weeks  from  the  date  of  receipt 
of  the  survey  to  respond. 
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Data  Presentation 


The  data  were  analyzed  using  the  Statistical  Package  for 
the  Social  Sciences  (SPSS)  and  the  SAS  System  for  Elementary 
Statistical  Analysis.  The  types  of  statistical  analyses  used 
were  previously  described  in  the  "Applying  the  Survey 
Instrument"  section.  All  statistical  tests  were  performed  at 
the  .05  level  of  significance.  This  level  of  significance 
was  chosen  because  equation  1  determines  the  sample  size  at 
the  .10  level  of  significance  for  proportionate  sampling. 
Since  this  research  did  not  use  proportionate  sampling,  the 
formula  provided  a  very  conservatively  large  sample  size. 
Therefore  the  sample  size  was  sufficient  to  perform  the  ANOVA 
at  the  .05  level  of  significance. 
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IV.  Findings 


Introduction 

The  survey  questionnaire  was  mailed  to  the  respondents 
at  CONUS  SAC  and  TAC  bases.  They  were  asked  to  complete  the 
questionnaire  and  return  it  within  two  weeks  of  the  date  of 
receipt.  A  total  of  872  surveys  were  mailed  by  31  May  1991 
and,  of  those  returned,  most  were  received  by  21  June  1991. 
Any  surveys  returned,  after  28  June  1991  were  not  included  in 
this  survey. 

The  respondents  used  a  computer  optical  scanning  form  to 
record  their  answers.  These  forms  were  automatically  read 
into  AFIT's  Academic  Computer  Support  System.  The  data  in 
this  research  was  analyzed  using  the  SAS  statistical  software 
system  and  is  presented  in  the  following  tables  and  graphs. 

Demographic  Data 

The  demographic  data  collected  by  the  survey  is 
presented  in  Tables  3  and  4. 


TABLE  3 

RETURN  RATE  OF  SURVEY  RESPONDENTS  BY  MAJCOM 


MAJCOM 

DISTRIBUTED 

RETURNED 

PERCENT 

TAC 

436 

180 

41.3 

SAC 

436 

191 

43.8 

TOTALS 

872 

371 

42.5 

Although 

these  response 

rates  were  less 

than  expected. 

they  were  still  considered  sufficient  for  generalizing  the 
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TABLE  4 


RETURN  RATE  OF  SURVEY  BY  RANK  SUBGROUP 


RANK 

DISTRIBUTED 

RETURNED 

PERCENT 

SAC 

Workers 

170 

52 

30.6 

Supervisors 

133 

74 

55.6 

Managers 

133 

65 

48.8 

TOTALS 

436 

191 

43.8 

TAC 

Subgroup  1 

170 

53 

31.2 

Subgroup  2 

133 

65 

48.8 

Subgroup  3 

133 

62 

46.6 

TOTALS 

436 

180 

41.3 

Workers  -  Airman  Basic  to  Staff  Sergeant 

Supervisors  -  Technical  Sergeant  to  Chief  Master  Sergeant 
Managers  -  2nd  Lieutenant  to  Colonel 

responses  to  the  general  population,  because  the  sample  size 
given  by  equation  1  was  conservatively  large. 

Multiple  Choice  Question  Responses 

Table  5  reflects  the  means  and  standard  deviations  of 
the  responses  for  questions  5  through  27  of  the  current 
research.  Table  6  reflects  the  results  of  the  two-way  ANOVA 
that  identified  any  statistically  significant  difference 
between  the  means  of  the  class  variables  (MAJCOM  and  Rank)  of 
the  current  research.  Table  7  indicates  by  which  level  of 
rank  the  results  were  significantly  different  according  to 
Bonferroni  (McClave  and  Benson:865).  There  were  no 
significant  interactions  between  MAJCOM  and  Rank  in  this 
ANOVA.  It  IS  interesting  to  note  from  Table  6  that  "rank" 
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TABLE  5 


MEANS 

AND  STANDARD 

DEVIATIONS  OF 

RESPONSES 

Question 

N 

Mean 

Std  Dev 

Q5 

370 

2.237 

0.866 

Q6 

369 

2.653 

0.985 

Q7 

367 

3.414 

1.095 

Q8 

369 

2.441 

1.046 

Q9 

370 

2.610 

1  . 156 

QIO 

369 

2.300 

0.637 

Qll 

366 

3.521 

0.958 

Q12 

359 

2.445 

1.078 

Q13 

364 

3.697 

1.927 

Q14 

367 

3.485 

2.429 

Q15 

368 

6.673 

3.090 

Q16 

367 

1.910 

1.301 

Q17 

367 

3.460 

2.019 

Q18 

366 

3.032 

1 . 910 

Q19 

365 

4.556 

2.512 

Q20 

329 

2.079 

2.129 

Q21 

364 

2.626 

1.575 

Q22 

365 

2.408 

1.458 

Q23 

365 

3.739 

1.884 

Q24 

364 

2.673 

1.617 

Q25 

363 

3.396 

1 .954 

Q26 

358 

2.122 

■  1.530 

Q27 

319 

1.413 

1.331 

explained  most  of  the  statistical  difference  identified  in 
this  current  research.  From  Table  7  it  is  also  notable,  for 
the  differences  in  rank  identified,  that  the  supervisors'  and 
the  managers'  responses  tended  to  be  different  as  well  as 
the  managers'  and  the  workers'  responses. 

Figures  1  through  15  present  the  distributions  of  the 
responses,  in  the  form  of  histograms,  for  questions  13-27  of 
the  current  research.  Survey  questions  13-27  attempted  to 
quantify  the  percentage  of  inaccurate  data,  the  percentage  of 
total  error  that  was  attributed  to  intentional  and  accidental 
causes,  and  the  specific  causes  of  intentional  and  accidental 
errors . 
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TABLE  6 


RESPONSES  COMPARED  BY  MAJCOM  AND  RANK 


QUES 

SAC 

TAC 

SGI 

SG2 

SG3 

COMMENT 

SIG  LEVEL 

Q5 

2.279 

2.191 

2.341 

2.360 

2.008 

b 

p<  .0019 

Q6 

2.768 

2.520 

2.631 

2.712 

2.617 

a 

p< .0158 

Q7 

3.421 

3.398 

3.155 

3.468 

3.547 

b 

p< . 0166 

Q8 

2.345 

2.537 

2.505 

2.266 

2.563 

b 

p<  .  0438 

Q9 

2.598 

2.674 

2.476 

2.863 

2.414 

b 

p< . 0033 

QIO 

2.288 

2.316 

2.291 

2.345 

2.242 

c 

Qll 

3.489 

3.545 

3.243 

3.338 

3.836 

b 

p<  .0001 

Q12 

2.328 

2.579 

2.379 

2.468 

2.344 

a 

p<  .  0250 

Q13 

3.596 

3.805 

3.845 

3.878 

3.227 

b 

p<  .0327 

Q14 

3.421 

3.556 

3.417 

3.971 

2.945 

b 

p< . 0030 

Q15 

6.888 

6.390 

6.631 

6.259 

7.000 

c 

Q16 

1.909 

1.915 

2.165 

1.971 

1.609 

b 

p< .0047 

Q17 

3.108 

3.787 

3.282 

3.266 

3.750 

a 

p<  .  0008 

Q18 

2.892 

3.165 

3.097 

2.950 

2.984 

c 

Q19 

4.476 

4.623 

4.330 

4.770 

4.328 

c 

Q20 

1.959 

2.225 

2.194 

1.950 

1.781 

c 

Q21 

2.494 

2.738 

2.670 

2.446 

2.703 

c 

Q22 

2.343 

2.471 

2.612 

2.367 

2.227 

c 

Q23 

3.447 

4.063 

3.981 

3.813 

3.352 

a 

p< .0014 

Q24 

2.602 

2.746 

3.049 

2.367 

2.617 

b 

p< .0018 

Q25 

3.220 

3.581 

3.194 

3.719 

3.055 

b 

p< .0145 

Q26 

2.000 

2.238 

2.272 

1.828 

2.165 

b 

p< .0499 

Q27 

1.343 

1.493 

1.534 

1.410 

1.156 

c 

SGI:  workers 

SG2:  supervisors 

SG3;  managers 
COMMENTS 

a:  The  means  differ 
b:  The  means  differ 

depending  on 
depending  on 

MAJCOM 

Rank  Subgroup 

c :  The  means 

do  not 

differ 

TABLE  7 

SUMMARY 

OF  DIFFERENCES  BY  RANK 

QUESTION 

MAN /WORK 

MAN/SUP  WORK/SUP 

Q5 

H 

* 

Q7 

♦ 

* 

Q8 

* 

Q9 

*  * 

Qll 

* 

* 

Q13 

* 

* 

Q14 

* 

Q16 

* 

* 

Q24 

* 

* 

Q25 

*  ilr 

Q26 

it 

* 

*  indicates  significant  difference  at  .05  level. 


37 


Referring  to  Figure  1,  it  is  evident  that  the  greatest 
number  of  maintenance  personnel  felt  that  10%  to  30%  of  the 
information  input  to  CAMS  was  inaccurate  and,  from  Figure  2 
that  0%  to  20%  of  the  total  errors  were  due  to  intentional 
causes.  However,  it  is  interesting  to  note  from  Figure  3 
that  there  was  not  as  much  agreement  on  how  much  of  the  total 
error  was  due  to  accidental  causes.  The  modal  response  for 
accidental  errors"  was  90%  or  more,  but  there  was  significant 
distribution  throughout  the  rest  of  the  percentages. 

In  attempting  to  quantify  the  causes  of  intentional 
errors,  it  is  evident  from  Figure  4  that  the  greatest  number 
of  maintenance  personnel  felt  that  only  0%  to  10%  of  the 
intentional  errors  were  due  to  pressure  to  falsify  the 
information.  Figure  5  indicates  that  "lack  of  adequate  time 
to  accurately  input  information"  accounted  for  0%  to  30%  of 
the  intentional  errors.  Additionally,  Figure  6  indicates 
that  the  greatest  number  of  maintenance  personnel  felt  that 
because  "personnel  do  not  perceive  any  benefit  from  entering 
accurate  information"  accounted  for  another  0%  to  30%  of  the 
intentional  errors.  It  is  clear  from  Figure  7  that  mainten¬ 
ance  personnel  disagreed  on  how  much  of  the  intentional 
errors  were  caused  by  the  "difficulty  in  entering  information 
into  CAMS,"  but  the  greatest  number  felt  that  it  was  10%  or 
more.  When  the  maintenance  personnel  were  asked  if  there 
were  any  causes  for  intentional  errors  that  were  not 
previously  listed.  Figure  8  indicates  that  the  respondents 
felt  there  were  few  other  causes.  Some  of  the  causes  that 
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QUESTION  13  RESPONSES 


12S4S6789  10 

ANSWra  NUMBER 


Figure  1 .  %  Inaccurate  Data 


QUESTION  1  4  RESPONSES 


125438789  10 

ANSWER  NIMER 


Figure  2.  ^  Intentional  Errors 


ANSWER  ANSWER  ANSWER  ANSWER 

1- 0%  4-30%  7=60%  10-90%  or  more 

2- 10%  5-40%  8-70% 

3- 20%  6-50%  9=80% 
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Figure  3.  %  Accidental  Errors 


ANSWER  ANSWER  ANSWER  ANSWER 

1- 0^  4=30^  7-60^  10-90^  or  more 

2- 10%  5-40%  8-70% 

3- 20%  6-50%  9-80% 
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QUESTION  17  RESPONSES 


100 
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Figure  5_  Lack  of  Adequate  Time 


QUESTION  18  RESPONSES 
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Figure  6.  No  Perception  of  Benefit 
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6- 50%  9=80% 
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Figure  0.  Other  Causes  of  Intentional  Errors 


ANSWER  ANSWER  ANSWER  ANSWER 

1“0%  4-30%  7=60%  10=90%  or  more 

2- 10%  5-40%  8=70% 

3- 20%  6=50%  9-80% 
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the  respondents  felt  were  significant,  but  were  not 
previously  listed  were  that:  "CAMS  won't  accept  the  correct 
information  so  erroneous  information  is  entered  to  clear  the 
job  (15  responses) .  personnel  are  lazy  and  not  motivated  to 
enter  the  correct  information  (7  responses),  and  personnel 
are  not  adequately  trained  on  CAMS  (8  responses) .  There  were 
several  other  miscellaneous  causes  that  had  five  responses  or 
less  that  are  not  listed.  These  may  be  obtained  from  the 
author  by  request . 

Figures  9  through  15  characterize  how  maintenance 
personnel  perceived  the  causes  of  accidental  errors.  Figure 
9  indicates  that  the  greatest  number  of  maintenance  personnel 
identified  that  keystroke  errors  accounted  for  10%  of 
accidental  errors.  From  Figure  10  it  is  apparent  that 
another  10%  of  accidental  errors  are  caused  by  "insufficient 
training  in  using  the  Technical  Order  (T.O.)  system."  A 
range  of  10%  to  50%  of  accidental  error  was  attributed  to 
"insufficient  training  in  using  the  CAMS  system"  as  indicated 
in  Figure  11.  Additionally,  it  is  clear  from  Figure  12  that 
the  greatest  number  of  maintenance  personnel  attributed 
another  10%  of  accidental  errors  to  "difficulty  in  finding 
the  correct  codes  in  the  T.O.  system  to  input  into  CAMS." 
Figure  13  indicates  that  the  greatest  number  of  maintenance 
personnel  attributed  0%  to  40%  of  accidental  errors  to  the 
"difficulty  in  using  the  multiple  screens  that  CAMS  requires 
for  data  entry."  Furthermore,  as  indicated  in  Figure  14, 
another  0%  to  10%  of  accidental  error  was  attributed  to 
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QUESTION  21  RESPONSES 


123436789  10 

ANSWER  NUMBER 


Figure  9 .  Keystroke  Errors 


QUESTION  22  RESPONSES 


123436789  10 

ANSWER  NIMCR 


Figure  10.  Insufficient  Training:  T.O.  Syst‘.?m 
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or  more 


ANSWER  ANSWER  ANSWER  ANSWER 

1- 0%  4-30%  7-60%  10=90%  or  more 

2- 10%  5-40%  8-70% 

3- 20%  6-50%  9-00% 
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QUESTION  25  RESPONSES 
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Figure  13.  Difficulty  in  Using  Multiple  Screens 


QUESTION  26  RESPONSES 
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Figure  14.  Computer  Malfunction 
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computer  malfunction.  Finally,  as  seen  in  Figure  15.  very 
few  maintenance  personnel  indicated  that  there  were  other 
reasons  not  previously  listed  for  causes  of  accidental 
errors.  There  were  no  consistent  responses  among  the  reasons 
cited  by  the  respondents  in  this  "other"  category. 


QUESTION  27  RESPONSES 
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Figure  15.  other  Reasons  for  Accidental  Errors 
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The  open  ended  questions  were  asked  to  further  the 
understanding  of  the  causes  of  errors  and  to  determine  what, 
if  anything,  could  help  to  make  the  CAMS  environment  less 
prone  to  errors.  The  responses  to  these  questions  were 
"categorized"  and  summarized  in  Figures  16  through  20.  In 
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the  following  presentation  of  these  results,  only  the  top  3 
categories  for  each  question  will  be  discussed  and  the 
remainder  are  included  for  inspection  in  appendix  C, 

Question  28  asked  "If  it  were  in  your  power  to  change 
the  CAMS  system  what  changes  would  you  make  and  why?"  The 
three  ways  in  which  the  greatest  number  of  maintenance 
technicians  said  that  they  would  change  were  to:  make  CAMS 
more  user  friendly  (1).  eliminate/reduce  the  number  of 
multiple  screen  entries  (2),  and  decrease  CAMS  downtime  (3). 


QUESTION  28  RESPONSES 


I  2  3  4  9  S  7  a  9  to  11  12  13  t4  IS 
/U6WER  NUMBER 

Figure  16.  Question  28  Responses 


Question  29  asked  "What  do  you  feel  is  the  single  most 
prevalent  reason  for  errors  occurring  in  the  CAMS  database?" 
The  top  3  items  that  the  greatest  number  of  maintenance 
personnel  cited  for  these  errors  were:  inadequate  training  on 
the  CAMS  system  (1),  personnel  are  not  motivated  to  make  the 
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1  2  3  i  5  6  7  8  9  to  11  12  13  1415  16  17 
ANSWER  NUMBER 


Figure  17.  Question  29  Responses 

correct  entries  (2).  and  CAMS  is  not  user  friendly  (3).  It 
is  interesting  to  note  from  Figure  17  that  inadequate 
training  is  by  far  the  largest  contributor  cited  by  the 
maintenance  personnel. 

Question  30  asked  "What  type  of  data  concerning 
maintenance  actions  are  most  often  reported  in  error?"  The 
top  3  items  cited  by  maintenance  personnel  for  these  errors 
were:  work  unit  codes  (1),  action  taken  codes  (2)  and  how 
malfunctioned  codes  (3). 

Question  31  asked  "If  you  know  any  person (s)  who 
intentionally  inputs  false  data  into  the  CAMS  system,  what 
are  the  top  three  most  common  reasons  given  for  their 
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Figure  18.  Question  30  Responses 


QUESTION  3 1  RESPONSES 
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Figure  19.  Question  31  Responses 
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actions?"  The  top  3  reasons  cited  by  maintenance  personnel 
were:  CAMS  won't  accept  the  correct  information,  so  something 
was  entered  to  clear  the  job  (1);  insufficient  time  to  make 
the  correct  entries  (2);  and  I  don't  know  anyone  who 
intentionally  inputs  incorrect  information  (3). 

Question  32  asked  "In  your  opinion,  what  is  the  single 
most  beneficial  action  we  could  take  to  reduce  or  eliminate 
data  entry  errors  in  CAMS?"  The  top  3  recommendations  the 
maintenance  personnel  made  were  to:  provide  training  on  the 
CAMS  system  (1),  make  CAMS  more  user  friendly  (2),  and 
include  on  line  help  capability  within  the  CAMS  system  (3). 


QUESTION  32  RESPONSES 


123496789  10 

AM5WER  NUMBER 


Figure  20.  Question  32  Responses 

Comparison  of  Current  Research  to  Folmar's  Research 

Responses  to  questions  5-11  and  14  were  used  to  directly 
compare  present  perceptions  of  data  inaccuracy  under  the  CAMS 
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environment  to  those  experienced  under  MDC  (Folmar ; 1986 ) . 

The  scales  for  these  questions  are  restated  here  for 
reference  in  the  following  results.  Questions  5-9  used  a 
Likert  scale  ranging  from  one  to  five  with  one  representing 
the  response  "strongly  agree"  and  five  representing  the 
response  "strongly  disagree." 

The  results  of  the  three-way  ANOVA  for  questions  5 
through  11  and  14  indicated  there  were  several  significant 
interactions  occurring.  An  approach  to  analyzing  these 
interactions  was  obtained  from  the  literature  (Mil  liken  and 
Johnson : 198-199 )  and  applied  to  the  results  of  the  ANOVA. 
These  interactions  are  summarized  in  Table  8  and  illustrated 
in  Figures  21  through  27.  The  only  question  that  did  not 
have  a  significant  interaction  term  was  question  7.  This  is 
the  only  question  where  the  main  effects  will  be  discussed. 

TABLE  8 

SUMMARY  OF  SIGNIFICANT  INTERACTIONS 


QUESTION 

INTERACTIVE 

TERM 

SIGNIFICANCE 

5 

Rank*Study 

p< .0015 

6 

Rank*Study 

p< . 0083 

7 

None 

None 

8 

Rank* Study 

p< .0463 

9 

Rank*Study 

p< .0143 

10 

Rank*Study 

p< .0354 

11 

Rank* Study 

p< .0001 

14 

Rank*Study 

p< .0001 

Note;  Tests  conducted  at  the  .05  level  of  significance. 

Question  5  examined  the  perception  of  "valuable 
information"  provided  under  both  the  CAMS  and  MDC 


Figure  21.  Q5  Rank*Study  Interaction 


environments.  Since  there  was  interaction  between  the 
variables  of  rank  and  study  in  the  analysis  of  this  question 
the  main  effects  between  command,  rank  and  study  will  not  be 
discussed.  Figure  5  displays  the  interaction  between  the 
variables  of  rank  and  study.  In  this  particular  case,  there 
is  no  statistically  significant  difference  between  the  two 
studies  at  the  worker  level  (p<.5746).  However,  at  the 
manager  (M)  and  Supervisor  (S)  levels  there  is  an  improvement 
(recall  that  the  further  toward  the  response  "one"  the  more 
in  agreement  the  respondent  was  with  the  statement.)  in  the 
perceived  value  of  information  provided  by  CAMS  over  MDC. 

Question  6  examined  the  perception  of  "timely  feedback" 
under  the  CAMS  and  MDC  environments.  Again  the  ANOVA 
revealed  interaction  between  the  variables  of  rank  and  study. 
Figure  22  shows  this  interaction.  At  the  manager  and 


supervisor  levels,  there  Is  improvement 'by  CAMS  over  the  MDC 
system  (p<.0002  and  p<.0001  respectively),  while  at  the 
worker  level  there  was  no  difference.  (p<.4037). 

Question  7  compared  the  perception  of  the  ability  to 
provide  accurate  accounting  of  manhour  utilization  under  the 
CAMS  and  MDC  environments.  This  was  the  only  comparison  in 
this  study  that  significant  interaction  effects  were  not 
observed.  The  perception  measured  under  MDC  (Folmar : 1986) 
produced  a  mean  of  3.636  and  the  perception  measured  in  this 
research  produced  a  mean  of  3.408.  This  result  was 
statistically  significant  at  p<.0018.  Although  this  is  an 
apparent  improvement  in  manhour  accounting  under  the  CAMS 
environment,  maintenance  personnel  under  both  systems  tended 
to  disagree  with  the  statement  that  the  respective  systems 
provided  accurate  manhour  accounting.  This  perception  was 
consistent  across  MAJCOM,  rank,  and  study. 
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Question  8  measured  the  perception  that  inaccurate  data 
is  input  to  the  system  because  of  the  difficulty  in  coding 
the  information  under  the  respective  CAMS  or  MDC  environment. 
The  ANOVA  revealed  interaction  between  the  rank  and  study 
variables.  Figure  23  shows  the  result  of  the  interaction. 

At  the  supervisor  level,  the  perception  increased  under  the 
CAMS  environment  (p<.0002)  while  at  the  manager  and  worker 
levels,  the  perception  has  not  significantly  changed. 


Question  9  evaluated  the  perception  that  the  system 
(CAMS  or  MDC)  was  a  valuable  management  tool  which  the  Air 
Force  should  retain.  Once  more  the  ANOVA  revealed 
interaction  between  the  rank  and  study  variables.  Figure  24 
shows  the  results  of  this  interaction.  The  perception 
improved  at  the  manager  level  (p<.0001)  but  did  not  improve 
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Figure  24.  Q9  Rank*Study  Interaction 


at  the  supervisor  or  worker  levels  (p<.4061  and  p<.2860 
respectively) . 

Questions  10  and  11  used  a  Likert  scale  ranging  from  one 
to  five  with  one  representing  the  response  "always"  and  five 
representing  the  response  "never." 

Question  10  evaluated  the  perception  that  correct  data 
is  input  in  the  system  (CAMS  or  MDC) .  Again,  there  was 
interaction  identified  between  the  variables  of  rank  and 
study.  Figure  25  shows  this  interaction.  At  the  manager  and 
supervisor  levels  there  is  an  improvement  under  the  CAMS 
environment  (p<.0001  and  p<.0011  respectively)  but  there  was 
no  improvement  at  the  worker  level  (p<.5224). 
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OETERMAN  FOLMAR 


Figure  25.  Qio  Rank*Study  Interaction 


Question  11  evaluated  the  perception  that  maintenance 
personnel  are  pressured  by  superiors  to  manipulate  the  data 
input  to  the  system  (CAMS  or  MDC) .  The  ANOVA  identified 
significant  interactions  between  the  rank  and  study  variables 
once  more.  This  interaction  is  shown  in  Figure  26.  There 
was  an  improvement  in  perception  at  the  manager  level 
(p<.0001)  but  there  was  no  significant  difference  at  the 
supervisor  or  worker  levels  {p<.3111  and  p.2418 
respectively) . 

Question  14  measured  the  perception  of  the  percentage  of 
inaccurate  information  that  was  input  into  the  system  (CAMS 
or  MDC)  by  using  a  scale  from  one  to  ten  representing 
percentages  from  0%  to  90%  or  more  respectively.  The  ANOVA 
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identified  interaction  between  the  rank  and  study  variables. 
This  interaction  is  shown  in  Figure  27.  Supervisors 
perceived  that  inaccurate  information  had  increased  under  the 
CAMS  environment  (p<.0004),  while  at  the  manager  level,  the 
perception  of  inaccuracy  had  decreased  under  the  CAMS 
environment  (p<.0133).  The  perception  at  the  worker  level 
was  not  significantly  different  (p<.3727). 

Summary 

The  results  presented  in  this  chapter  characterized  the 
nature,  extent  and  causes  of  data  inaccuracy  under  the  CAMS 
environment.  Additionally,  the  results  indicate  that  the 
rank  of  maintenance  personnel  influences  their  perception  of 
data  inaccuracy.  Also,  the  maintenance  personnel  indicated 
that  the  largest  contributor  to  data  inaccuracy  was  lack  of 
training  on  the  CAMS  system  and  that  a  training  program  could 
aid  in  reducing  data  inaccuracy.  Finally,  the  results 
indicate  that,  in  general,  there  was  perceived  improvement  in 
data  accuracy  under  the  CAMS  environment  at  the  manager  and 
supervisor  levels,  but  not  at  the  worker  level. 


59 


V.  Analysis.  Conclusions,  and  Recommendations 


Meeting  the  Research  Objectives 

This  research  fulfilled  the  three  primary  objectives 
stated  in  Chapter  1.  It  examined  the  extent,  nature,  and 
causes  of  data  inaccuracies  being  input  into  CAMS. 
Additionally,  it  examined  whether  perceptions  of  data 
inaccuracies  varied  across  MAJCOMS  and  rank  structure. 
Finally,  it  examined  the  changes,  if  any.  in  the  perception 
of  data  inaccuracy  under  the  CAMS  environment  when  compared 
to  the  previous  MDC  environment  (Folmar ; 1986) ,  By  restating 
the  research  objectives  here,  a  logical  framework  is  provided 
to  analyze  the  results  obtained  in  this  research. 

First  Objective.  The  research  identified  the  extent, 
nature,  and  causes  of  data  inaccuracies  being  input  into 
CAMS.  To  accomplish  this  analysis,  the  means  introduced  in 
Table  5  of  Chapter  4  were  converted  to  the  percentages 
corresponding  to  the  scale  used  in  the  particular  survey 
questions.  For  example,  in  question  14  a  mean  of  3.485  was 
reported.  This  translates  to  an  approximate  mean  of  25^. 
These  percentages  were  used  in  the  following  analysis  to 
supplement  the  modal  analysis.  The  modal  responses  and 
percentages  for  each  question  are  summarized  in  Tables  9  and 
10. 

The  results  from  survey  question  13  were  used  to  examine 
the  "extent"  of  data  inaccuracy  under  the  CAMS  environment. 
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TABLE  9 


SUMMARY  OF 

THE  NATURE  OF  DATA  INACCURACY 
IN  THE  CAMS  ENVIRONMENT 


QUESTION 

NATURE 

MODE(%) 

MEAN(%) 

Q13 

% 

Inaccurate  Info 

10 

27 

Q14 

% 

Intentional  Errors 

10 

25 

Q15 

% 

Accidental  Errors 

57 

TABLE  10 

SUMMARY 

OF 

THE  CAUSES  OF  DATA 

INACCURACY 

IN  THE  CAMS  ENVIRONMENT 

QUESTION 

CAUSE 

MODE(%) 

MEAN(%) 

INTENTIONAL  : 

ERRORS 

Q16 

Pressure  to  Falsify 

0 

9 

Q17 

Lack  of  Time 

10 

25 

Q18 

No  Perception  of 
Benefit 

10 

20 

Q19 

Difficulty  in 

Entering  the  Data 

10 

36 

Q20 

Other  Causes 

_0 

n 

Totals 

30 

101 

ACCIDENTAL  ERRORS 

.  Q21 

Keystroke  Errors 

10 

16 

Q22 

Insufficient  Training: 
T,0.  System 

10 

14 

Q23 

Insufficient  Training: 
CAMS  System 

10 

27 

Q24 

Difficulty  in  Locating 
Correct  Codes  in  T.O.s 

10 

17 

Q25 

Difficulty  in  Using 
Multiple  Screen  Entry 

10 

24 

Q26 

Computer  Malfunction 

0 

11 

Q27 

Other  Causes 

_0 

_4 

Totals 

50 

113 

The  results  reflect  that,  on  average  (the  mean  response),  the 
maintenance  personnel  feel  27%  of  the  information  input  into 
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CAMS  is  inaccurate.  The  greatest  number  of  maintenance 
personnel  (the  modal  response)  felt  that  \Q%  of  the 
information  input  into  CAMS  is  inaccurate. 

The  "nature"  of  data  inaccuracy  is  indicated  by  the 
results  from  questions  14  and  15.  The  results  indicate  that, 
on  average,  maintenance  personnel  felt  25%  of  the  total  error 
was  intentional,  while  the  greatest  number  of  maintenance 
personnel  felt  10%  of  the  total  error  was  intentional.  The 
results  also  indicated  that,  on  average,  maintenance 
personnel  felt  57%  of  the  erroneous  information  was  input 
into  CAMS  accidentally,  while  the  greatest  number  of 
maintenance  personnel  felt  that  90%  or  more  of  the  errors  are 
due  to  accidental  causes.  Note  here  that  the  sum  of  the 
"averages"  (25%  and  57%)  for  questions  14  and  15  did  not 
quite  add  to  100%,  but  the  modal  responses  do.  However,  as 
Figure  3  indicates,  the  distribution  of  responses  to  question 
15  was  "bowl"  shaped,  suggesting  the  presence  of  divided 
opinion.  Additionally,  the  respondents  may  not  have  ensured 
that  their  responses  for  questions  14  and  15  added  to  100%. 

The  "causes"  of  intentional  errors  were  indicated  by  the 
results  of  survey  questions  16  through  20.  The  maintenance 
personnel  felt  that,  on  average:  09%  of  the  intentional 
errors  were  due  to  pressure  to  falsify /misrepresent  the 
information,  25%  of  the  intentional  errors  were  due  to  laclc 
of  adequate  time  to  accurately  input  information.  20%  of  the 
intentional  errors  were  due  to  personnel  not  perceiving  any 
benefit  from  entering  accurate  information,  36%  of  the 
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intentional  errors  were  due  to  the  difficulty  in  entering  the 
data,  and  11%  were  due  to  other  reasons. 

The  "causes"  of  accidental  errors  were  indicated  by  the 
results  of  survey  questions  21  through  27.  The  maintenance 
personnel  felt  that,  on  average:  16%  of  the  accidental  errors 
were  due  to  keystroke  errors.  14%  of  the  accidental  errors 
were  due  to  insufficient  training  on  the  T.O.  system, '27%  of 
the  accidental  errors  were  due  to  insufficient  training  on 
the  CAMS  system,  17%  of  the  accidental  errors  were  due  to 
difficulty  in  finding  the  correct  codes  in  the  T.O.  system  to 
input  into  CAMS,  24%  of  the  intentional  errors  were  due  to 
the  difficulty  in  using  the  multiple  screens  that  CAMS 
requires  for  certain  types  of  data  entry,  11%  of  the 
accidental  errors  were  due  to  computer  malfunction,  and  4%  of 
the  accidental  errors  were  due  to  other  causes.  Note  here 
that  the  sum  of  the  percentages  for  the  accidental  errors  is 
113%.  However,  due  to  the  number  of  the  possible  causes 
provided  in  the  survey  and  the  standard  deviation  for  each 
cause,  it  was  felt  that  the  means  of  the  responses  for  the 
survey  questions  still  provided  an  accurate  characterization 
of  the  contribution  to  accidental  errors  by  each  cause. 

Second  Objective.  Identify  any  variance  of  the 
responses  across  MAJCOMS  or  rank  structure.  To  accomplish 
this  objective,  a  two-way  ANOVA  was  conducted  on  the 
responses  of  the  current  research.  Tables  6  and  7,  presented 
in  chapter  4,  display  the  results  of  the  ANOVA.  Table  11 
summarizes  these  results  by  MAJCOM  and  rank.  It  is 
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TABLE  11 


SUMMARY 

OF  RESPONSE 

VARIANCE 

BY  MAJCOM 

AND  RANK 

QUESTION 

MAJCOM 

RANK 

NO  VAR 

SIG  LEVEL 

Q5 

* 

p<  .  0019 

Q6 

p< .0158 

Q7 

* 

p<  .0166 

Q8 

* 

p<  .0438 

Q9 

* 

p< . 0033 

QIO 

k 

Qll 

* 

p< . 0001 

Q12 

* 

p<  .  0250 

Q13 

* 

p< . 0327 

Q14 

•k 

p< . 0030 

Q15 

k 

Q16 

k 

p< .0047 

Q17 

* 

p<  .  0008 

Q19 

k 

Q20 

k 

Q21 

k 

Q22 

k 

Q23 

* 

p<  .0014 

Q24 

k 

p< . 0018 

Q25 

k 

p< .0145 

Q26 

k 

p< . 0499 

Q27 

k 

interesting  to  note,  from  Table  11,  that  there  was 
significant  variance  in  this  analysis  and  that  most  of  the 
variance  is  explained  by  rank.  However,  there  was  some 
variance  explained  by  MAJCOM  which  will  be  analyzed  first. 

The  first  case  where  the  responses  differed  by  MAJCOM  is 
question  6.  The  SAC  maintenance  personnel  disagreed  more  than 
TAC  maintenance  personnel  with  the  statement  that  CAMS 
provides  timely  feedback  for  the  base-level  managers. 

However,  both  groups  tended  to  agree  with  that  statement. 
Also,  as  indicated  in  the  question  12  responses,  the  SAC 
maintenance  personnel  tended  to  think  that  the  majority  of 
inaccurate  data  that  is  input  to  the  CAMS  system  was  caused 
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by  errors  in  providing  the  data  required  by  the  CAMS  system. 
The  TAC  maintenance  personnel  felt  that  the  majority  of 
inaccurate  data  that  is  input  into  the  CAMS  system  was  caused 
by  manipulation  of  the  data  to  meet  expectations. 
Additionally,  as  indicated  in  the  question  17  responses,  the 
SAC  maintenance  personnel  tended  to  feel  that  the  lack  of 
adequate  time  to  accurately  input  information  to  CAMS 
accounted  for  30%  of  the  intentional  errors  input  to  CAMS 
while,  the  TAC  maintenance  personnel  felt  this  reason  for 
intentional  errors  accounted  for  only  20%.  Finally,  as 
indicated  from  the  question  23  responses.  SAC  maintenance 
personnel  tended  to  feel  that  insufficient  training  on  the 
CAMS  system  accounted  for  20%  of  the  accidental  errors  input 
into  CAMS,  while  TAC  maintenance  personnel  felt  that  this 
cause  accounted  for  30%  of  the  accidental  errors  input  into 
CAMS. 

In  the  cases  where  the  difference  between  the  responses 
were  accounted  for  by  rank,  as  seen  from  Table  7,  the 
maintenance  personnel  at  the  manager  level  tended  to  ditfer 
from  the  supervisors  and  workers  in  their  perception  of  data 
inaccuracy.  As  seen  from  the  results  of  question  5,  the 
managers  agreed  more  than  the  workers  or  supervisors  with  the 
statement  that  CAMS  provides  valuable  information  for  base- 
level  maintenance  managers.  In  responding  to  question  7,  the 
workers  agreed  more  than  the  supervisors  or  managers  with  the 
statement  that  the  CAMS  system  provides  an  accurate 
accounting  of  manhour  utilization  in  the  maintenance  complex. 
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In  answering  question  8,  the  supervisors  agreed  more  than  the 
managers  with  the  statement  that  inaccurate  data  is  input  to 
the  CAMS  due  to  the  difficulty  in  accurately  coding  the 
information  required  by  the  CAMS  system.  (The  perceptions 
between  the  supervisors  and  workers  for  this  statement  were 
not  statistically  significant  at  the  .05  level.)  Also,  in 
responding  to  question  9.  the  supervisors  disagreed  more  than 
the  managers  or  workers  with  the  statement  that  CAMS  is  a 
useful  management  tool  which  should  be  maintained  by  the  Air 
Force.  The  managers  in  responding  to  question  11  felt  that 
maintenance  personnel  are  seldom  pressured  by  superiors  to 
manipulate  the  CAMS  input,  while  the  supervisors  and  workers 
felt  that  the  pressure  to  manipulate  the  data  was  somewhat 
higher.  In  question  13,  managers  felt  there  was  a  lower 
percentage  of  inaccurate  information  input  info  CAMS  than 
that  perceived  by  supervisors  and  workers.  Question  14 
reveals  that  the  supervisors  perceived  a  higher  percentage  of 
intentional  errors,  as  a  portion  of  the  total  errors,  than 
the  managers.  (The  difference  in  perceptions  between  the 
supervisors  and  workers  for  this  question  was  not 
statistically  significant  at  the  .05  level.)  Question  16 
reveals  that  the  workers  and  supervisors  felt  the  pressure  to 
falsify/manipulate  information  was  greater  than  that 
perceived  by  the  managers.  In  responding  to  question  24.  the 
workers  perceived  that  the  difficulty  in  finding  the  correct 
codes  in  the  T.O.  system  to  input  to  CAMS  accounted  for  a 
greater  percentage  of  the  accidental  errors  than  was 
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perceived  by  the  supervisors  and  managers.  Additionally,  the 
responses  to  question  25  reveal  that  the  supervisors 
perceived  a  higher  percentage,  than  was  noted  by  the 
supervisors  or  workers,  of  the  accidental  errors  that  were 
due  to  the  difficulty  in  using  the  multiple  screens  CAMS 
requires  for  certain  entries.  Finally,  in  question  26,  the 
managers  felt  that  the  percentage  of  accidental  errors 
explained  by  computer  malfunction  was  less  than  that 
perceived  by  the  supervisors  or  workers. 

Third  Objective.  Identify  any  changes  in  the 
perceptions  of  data  inaccuracy  under  the  CAMS  environment 
when  compared  to  the  earlier  MDC  environment  (Folmar : 1986) . 

To  accomplish  this  objective,  a  three-way  ANOVA  was  conducted 
for  each  of  the  survey  questions  5-11  and  14  to  compare  the 
results  of  the  current  research  to  the  same  survey  questions 
asked  by  Folmar  (Folmar : 1986) .  The  results  of  the  ANOVA 
revealed  that  there  was  a  significant  rank*study  interaction 
for  all  of  the  survey  questions  except  question  7. 

Survey  question  7  examined  the  perception  that  the 
system  (CAMS  or  MDC)  provided  an  accurate  accounting  of 
manhour  utilization  in  the  maintenance  complex.  There  is  no 
difference  in  the  perception,  based  on  MAJCOM  (p<.4260),  of 
SAC  personnel  (mean  response=3 . 571 )  and  TAC  personnel  (mean 
response-3.512)  in  their  response  to  this  question.  Both 
slightly  disagreed  with  the  statement  that  the  respective 
systems  (CAMS  or  MDC)  provide  an  accurate  accounting  of 
maintenance  manhours.  There  was  a  significant  difference 
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based  on  rank  (p<.0001).  in  that  managers  (mean  response 
-3.651)  and  supervisors  (mean  response-3.630)  disagreed  more 
than  the  workers  (mean  response=3 . 300 )  with  the  statement 
that  the  systems  (CAMS  or  MDC)  provided  an  accurate 
accounting  of  manhour  utilization  in  the  maintenance  complex. 
There  was  also  a  significant  difference  based  on  study 
(p<.0018).  Maintenance  personnel  felt  CAMS  (mean 
response-3.408)  provided  a  more  accurate  accounting  of 
manhour  utilization  in  the  maintenance  complex  than  did  the 
MDC  system  (mean  response-3 . 637) . 

Figures  21  through  27,  presented  in  Chapter  4,  display 
the  interactions  observed  in  survey  questions  5,  6  ,  8-11, 
and  14,  In  each  question  there  was  significant  rank*study 
interaction.  Table  12  summarizes,  by  rank,  the  significant 
differences  in  perceptions  identified  between  MDC  and  CAMS 
using  Bonferroni's  procedure  (McClave  and  Benson:865). 


TABLE  12 

DIFFERENCES  IN  PERCEPTION  IDENTIFIED  BETWEEN 
MDC  AND  CAMS  ENVIRONMENT  BY  RANK 


QUESTION 

MANAGERS 

SUPERVISORS 

WORKERS 

Q5 

p<  .0001 

p<  .0009 

not  sig 

Q6 

p<  .0002 

p<  .0001 

not  sig 

Q8 

not  sig 

p<.0002 

not  sig 

Q9 

p< .0001 

not  sig 

not  sig 

QIO 

p< . 0001 

p< .0011 

not  sig 

Qll 

p< .0001 

not  sig 

not  sig 

ql4 

p<  .0004 

p< .0133 

not  sig 

From 

Table  12,  it  is 

interesting  to  note 

that  the 

responses 

to  the  questions 

at  the  worker  level 

did  not  change 
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between  the  two  studies.  However,  in  general  for  questions 
5,  6.  and  10.  managers  and  supervisors  perceive  that  the  data 
collection  environment  has  improved  under  CAMS.  In 
responding  to  question  8,  as  shown  in  Figure  23,  only  the 
supervisors  agreed  more  with  the  statement  that,  under  the 
CAMS  environment,  inaccurate  data  is  input  into  the  system 
due  to  the  difficulty  in  accurately  coding  the  information 
required  by  the  system.  Additionally,  Figure  24  reflects 
that,  in  answering  question  9,  only  the  managers  agreed  more 
with  the  statement  that  CAMS  is  a  useful  management  tool  that 
should  be  retained  by  the  Air  Force.  In  responding  to 
question  11,  only  the  managers  felt  there  was  less  pressure, 
under  the  CAMS  environment,  by  superiors  on  subordinates  to 
manipulate  the  data  input.  Finally,  Figure  27  indicates  a 
unique  interaction.  The  managers  felt  that  the  percentage  of 
inaccurate  data  that  is  due  to  intentional  errors  decreased 
under  the  CAMS  environment  while  the  supervisors  perceived 
that  it  increased.  The  workers  did  not  perceive  any 
significant  change  under  the  CAMS  environment. 

Conclusions 

This  current  research  continued  the  investigation 
initiated  by  Capt  Folmar  (Folmar : 1986) .  It  identified  the 
nature  of  data  inaccuracy  under  the  CAMS  environment  by 
categorizing  the  inaccuracy  into  intentional  and  accidental 
causes.  The  modal  responses  to  questions  asking  the 
maintenance  technicians  to  quantify  the  errors  in  each  of 
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these  categories  indicated  that  10%  of  the  data  inaccuracies 
were  due  to  intentional  causes  while  90%  were  accidental. 
Therefore,  the  evidence  suggests  that  these  categories 
adequately  described  the  data  errors*  "universe.*' 

The  responses  to  the  causes  presented  to  the  maintenance 
technicians  indicated  that  this  research's  initial  attempt  to 
categorize  the  possible  causes  for  data  inaccuracies  was 
successful .  This  is  supported  by  the  fact  that  the 
respondents  indicated  very  few  other  causes  for  intentional 
or  accidental  errors.  This  is  not  to  say  that  other  causes 
don't  exist,  but  rather  their  impact  on  the  causes  already 
identified  would  probably  be  negligible.  Additionally,  the 
results  indicate  that  the  highest  percentage  of  intentional 
errors  (35%)  were  due  to  the  difficulty  in  entering 
information  into  CAMS,  Also,  the  results  suggest  that  the 
highest  percentage  of  accidental  errors  were  due  to 
insufficient  training  in  using  the  CAMS  system  (27%)  and 
difficulty  in  using  the  multiple  screens  that  CAMS  requires 
for  certain  types  of  data  entry  (24%). 

Finally,  in  examining  the  possible  explanations  for 
variance  in  the  ANOVA  conducted  in  this  research,  rank  was  a 
significant  contributing  factor  in  all  areas,  especially  in 
comparing  the  current  research  to  Folmar's  research 
(Folmar : 1986) .  In  this  last  case,  without  exception,  the 
workers  did  not  perceive  any  change  in  data  inaccuracy  under 
the  CAMS  environment.  This  should  be  of  special  concern 
because  they  are  the  people  entering  the  data  and  an 
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explanation  of  why  they  did  not  perceive  any  change  should  be 
pursued.  To  facilitate  this,  several  different  lines  of 
thought  are  posed,  for  further  reflection,  in  the  form  of 
questions.  First,  are  the  managers  and  supervisors  not  aware 
of  what  is  really  being  input  into  the  maintenance 
information  systems  by  the  workers?  Alternatively,  was  the 
environment,  under  the  MDC  system,  not  as  the  managers  and 
supervisors  perceived  it  to  be  at  the  time  of  Folmar's  study? 
Finally,  is  it  possible  that  the  managers  and  supervisors 
correctly  perceived  a  change  which  went  unnoticed  by  the 
workers? 

Recommendations 

The  evidence  presented  in  the  responses  to  the  open- 
ended  questions  suggest  some  courses  of  action  that  may 
improve  the  CAMS  data  collection  environment. 

First,  training  is  clearly  perceived  to  be  needed.  The 
evidence  gathered  in  this  current  research  suggests,  as 
indicated  in  the  responses  to  survey  question  29,  inadequate 
training  on  CAMS  to  be  the  single  most  prevalent  reason  for 
errors  occurring  in  the  CAMS  database.  Additionally,  when 
asked  what  they  felt  was  the  single  mos.t  beneficial  action 
that  could  be  taken  to  reduce  or  eliminate  data  entry  errors 
in  the  CAMS  system,  the  number  one  response  was  to  provide 
more  quality  training  on  the  CAMS  system.  The  perception  of 
a  training  deficiency  on  the  CAMS  system  suggests  a  good  area 
for  further  research:  identifying  what  type  of  training  is 
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needed  on  CAMS  and  determining  whether  the  training 
requirements  are  different  across  ranks  and  MAJCOMS. 

The  results  of  the  current  research  also  indicate  that 
investigation  is  warranted  concerning  how  to  make  the  CAMS 
data  collection  environment  more  "user  friendly."  The  number 
one  response  to  question  28  and  the  number  2  response  to 
question  32  were  to  make  CAMS  more  "user  friendly."  Both  of 
these  questions  asked  what  the  maintenance  personnel  would  do 
to  improve  the  data  collection  environment.  There  were 
several  comments  about  what  "user  friendly"  features  should 
be  included  with  the  most  popular  being  "pull  down  menus", 
extensive  "on-line  help"  functions,  and  elimination/reduction 
of  multiple  screen  data  entry  requirements.  The 
determination  of  requirements,  costs,  and  trade-offs  in  this 
area  is  rich  in  its  potential  for  future  research. 

One  last  area  that  is  recommended  for  future  research  is 
assessment  of  the  impact  of  data  inaccuracies  occurring  under 
the  CAMS  data  collection  environment.  This  research 
quantified  a  general  perception  that  errors  are  occurring, 
but  it  was  beyond  the  scope  of  the  research  to  quantify  what 
happens  after  the  incorrect  information  is  processed  through 
the  system.  This  line  of  investigation  could  include  impacts 
at  the  base,  MAJCOM,  and  Air  Force  levels. 
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Appendix  A;  Survey  Package 


LS  (Capt  Jon  Determan.  AV  785-8989) 

Core  Automated  Maintenance  System  Survey  Package 


Survey  Respondents 

1.  Please  take  the  time  to  complete  the  attached 
questionnaire  and  return  it  to  us  in  the  enclosed  envelope 
within  two  weeks. 

2.  The  survey,  SCN  91-24,  measures  your  perceptions  and 
attitudes  toward  the  current  Core  Automated  Maintenance 
System  (CAMS).  The  data  we  gather  will  become  part  of  an  Air 
Force  Institute  of  Technology  (AFIT)  research  project  and  may 
influence  the  design  of  future  data  collection  systems. 

3.  Your  individual  response  will  be  combined  with  others  and 
will  NOT  be  attributed  to  you  personally.  Your  identity  will 
remain  completely  anonymous. 

4.  Your  participation  in  this  research  effort  is  completely 
voluntary,  but  we  would  certainly  appreciate  your  help. 


PAUL  T.  WELCH,  Colonel,  USAF 
Associate  Dean 

School  of  Systems  and  Logistics 


1  Atch 

Questionnaire 


SCN  91-24 

Exp  date:  1  Aug  91 

INSTRUCTIONS 


1.  Do  not  write  your  name  or  your  social  security  number  on 
the  survey  questionnaire.  All  replies  will  be  completely 
anonymous . 

Please  enter  your  AFSC.  left  justified,  in  the  first  5  (or  4 
spaces  for  officers)  spaces  of  the  identification  number 
block  in  the  bottom  left  hand  corner  of  the  answer  form 
provided  with  the  survey.  Please  use  a  "soft  lead"  (No.  2) 
pencil,  and  observe  the  following: 

a.  Make  heavy  black  marks  that  fill  in  the  space  of  the 
response  you  select. 

b.  Erase  cleanly  any  responses  you  wish  to  change. 

c.  Make  no  stray  marks  of  any  kind  on  the  response 
sheet . 

d.  Do  not  staple,  fold  or  tear  the  response  sheet. 

2.  Read  all  questions  carefully.  Mark  the  answer,  on  the 
answer  sheet  provided,  that  best  describes  your  opinion  on 
the  question  asked.  Answer  all  questions  to  the  best  of  your 
knowledge . 

3.  On  the  open  ended  questions,  write  your  answers  in  the 
space  provided.  If  more  space  is  required,  please  attach  any 
additional  sheets  required. 

4.  Upon  completion,  please  place  your  survey  and  answer 
sheet  in  the  enclosed  return  envelope  and  place  the  envelope 
in  base  distribution. 

5.  Please  try  to  return  the  survey  within  two  weeks  so  that 
we  may  get  the  results  published  as  soon  as  possible. 

6.  Thank  you  for  your  time  and  cooperation. 
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Section  I.  Background  Information 

This  section  contains  several  items  dealing  with 
personal  characteristics.  This  information  will  be  used  to 
describe  the  population  of  the  study.  If  you  feel  a 
requested  item  of  information  may  specifically  identify  you, 
please  leave  that  item  blank. 

1.  You  are  assigned  to  which  major  command? 

(1)  Tactical  Air  Command  (TAG) 

(2)  Strategic  Air  Command  (SAC) 

(3)  Military  Airlift  Command  (MAC) 

(4)  United  States  Air  Force  Europe  (USAFE) 

(5)  Pacific  Air  Force  (PACAF) 

2.  Your  rank  is? 

(1)  AB  -  SSgt  (2)  TSgt  -  CMSgt  (3)  2Lt  -  Col 

3.  How  many  full  years  have  you  worked  in  the  aircraft 
maintenance  field? 

(1)  Less  than  2  years 

(2)  2  but  less  than  5  years 

(3)  5  but  less  than  10  years 

(4)  10  but  less  than  15  years 

(5)  15  but  less  than  20  years 

(6)  20  years  or  more 

4.  How  many  full  years  have  you  worked  with  the  CAMS 
system? 

(1)  Less  than  2  years 

(2)  2  but  less  than  5  years 

(3)  5  years  or  more 

Section  II.  Please  respond  to  the  following  statements  as 
honestly  and  openly  as  possible.  Use  the  following  scale  to 
answer  questions  5-9. 

(1) - (2) - (3) - (4) - (5) 

Strongly  Agree  Neither  Agree  Disagree  Strongly 

Agree  nor  Disagree  Disagree 

5.  CAMS  provides  valuable  information  for  base-level 
maintenance  managers. 

6.  The  CAMS  system  provides  timely  feedback  for  base- 
level  maintenance  managers. 

7.  The  CAMS  system  provides  an  accurate  accounting  of 
man-hour  utilization  in  the  maintenance  complex. 

0.  Inaccurate  data  is  input  to  the  CAMS  due  to  the 
difficulty  in  accurately  coding  the  information  required  by 
the  CAMS  System. 
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9.  The  CAMS  system  is  a  useful  management  tool  which 
should  be  retained  by  the  Air  Force. 

10.  Correct  data  is  input  in  the  CAMS  system  by 
maintenance  personnel: 

(1)  (2)  (3)  (4)  (5) 

Always  Usually  Sometimes  Seldom  Never 

11.  Maintenance  personnel  are  pressured  by  superiors  to 
manipulate  the  CAMS  input: 

(1)  (2)  (3)  (4)  (5) 

Always  Usually  Sometimes  Seldom  Never 

12.  The  majority  of  inaccurate  data  that  is  input  to 
the  CAMS  system  is  caused  by 

(1)  Keystroke  errors 

(2)  Error  in  providing  the  data  required  by  the  CAMS 
system  (examples:  work  unit  codes,  how 
malfunctioned  codes,  action  taken  codes,  job 
control  numbers  etc...) 

(3)  Manipulation  of  the  input  to  meet  expectations 

(4)  Computer  malfunctions 

(5)  Other  (specify) _ 

Section  III.  Information  Inaccuracy. 

For  the  remainder  of  the  survey,  the  questions  consider 
information  entered  at  the  data  element  level.  A  data  element 
is  an  item  specifically  identified  by  a  field  on  the  CAMS 
screen  (ie...Work  Unit  Code.  Job  Control  Number,  How  Mai  Code 
etc . . , ) . 


Use 

the 

following  scale  to 

answer 

questions  13-15. 

Special 

Note : 

Total  for  questions  14 

and  15  must  equal  100% 

(1) 

0% 

(6) 

50% 

(2) 

10% 

(7) 

60% 

(3) 

20% 

(8) 

70% 

(4) 

30% 

(9) 

80% 

(5) 

40% 

(10) 

90%  or  more 

13. 

In 

your  opinion,  what 

percentage  of  the  data  input 

into  the  CAMS  system  is  inaccurate  concerning  maintenance 
actions  that  occurred? 

14.  What  percentage  of  the  inaccurate  data  is  due  to 
intentional  errors?  (Definition:  Data  the  technician  knows 
is  incorrect  at  the  time  of  entry.) 
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15.  What  percentage  of  the  inaccurate  data  is  due  to 
accidental  errors?  (Definition:  Data  the  technician  does 

not  know  is  incorrect  at  the  time  of  entry.) 

Use  scale  below  to  answer  questions  16-20  to  indicate 
what  percent  of  total  intentional  error  is  caused  by  each  of 
the  items  listed.  Special  Note:  Total  for  questions  16-20 
must  equal  100%. 


(1) 

0% 

(6) 

50% 

(2) 

10% 

(7) 

60% 

(3) 

20% 

(8) 

70% 

(4) 

30% 

(9) 

80% 

(5) 

40% 

(10) 

90%  or  more 

16.  Pressure  to  falsify/misrepresent  information. 

17.  Lack  of  adequate  time  to  accurately  input 
information. 

18.  Personnel  do  not  perceive  any  benefit  from  entering 
accurate  information. 

19.  Information  is  difficult  to  enter.  (example:  The 
number  of  screens  CAMS  requires  to  complete  S/N  and  P/N  time 
change  items.) 

20.  Other.  Please  specify  reason  on  this  sheet  and 
mark  percentage  (from  above  scale)  on  answer  sheet. 


Use  scale  below  to  answer  questions  21-27  to  indicate 
what  percent  of  total  accidental  error  is  caused  by  each  of 
the  items  listed.  Special  Note:  Total  for  questions  21-27 
must  equal  100%. 
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(7) 
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(3) 

20% 

(8) 

70% 

(4) 

30% 

(9) 

80% 

(5) 

40% 

(10) 

90%  or  more 

21.  Keystroke  errors. 

22.  Insufficient  training  in  using  the  T.O.  system. 

23.  Insufficient  training  in  using  the  CAMS  system. 

24.  Difficulty  in  finding  the  correct  codes  in  the  T.O. 
system  to  input  into  CAMS. 

25.  Difficulty  in  using  the  multiple  screens  that  CAMS 
requires  for  certain  types  of  data  entry. 
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26 .  Computer  Ma 1 f unct i on , 

27.  Other.  Please  specify  reason  on  this  sheet  and 
mark  percentage  (from  above  scale)  on  answer  sheet. 


Open  Ended  Questions 


28.  If  it  were  in  your  power  to  change  the  CAMS  system, 
what  changes  would  you  make  and  why?  PLEASE  BE  AS  SPECIFIC 
AS  POSSIBLE. 


29.  What  do  you  feel  is  the  single  most  prevalent 
reason  for  errors  occurring  in  the  CAMS  data  base? 


30.  What  type  of  data  concerning  maintenance  actions  is 
most  often  reported  in  error? 


31.  If  you  know  any  person(s)  who  intentionally  inputs 
false  data  into  the  CAMS  system,  what  are  the  top  3  most 
common  reasons  given  for  their  actions? 


32.  In  your  opinion,  what  is  the  single  most  beneficial 
action  we  could  take  to  reduce  or  eliminate  data  entry  errors 
in  CAMS? 


Please  feel  free  to  use  the  remaining  space  to  comment 
on  the  advantages,  disadvantages,  problems,  or  other 
important  aspects  of  the  CAMS  system  as  you  perceive  it. 


Appendix  B:  Data  Table 


190 

4.5274 

189 

45274 

188 

4024 

187 

45650 

186 

45770 

185 

46250 

184 

46250 

183 

45200 

182 

45471 

181 

45470 

180 

45551 

179 

4024 

178 

45274 

177 

46290 

176 

45273 

175 

45770 

174 

45470 

173 

45451 

172 

45471 

171 

46290 

170 

45872 

169 

45474 

168 

4096 

167 

45670 

166 

45491 

165 

4016 

164 

45571 

163 

162 

45474 

161 

45474 

160 

4024 

159 

45451 

158 

45174 

157 

45400 

156 

45799 

155 

45770 

154 

45451 

153 

45274 

152 

45274 

151 

150 

45553 

149 

45651 

148 

45431 

147 

45651 

146 

45274 

145 

46255 

144 

4016 

143 

142 

45851 

0 1 1 i 1 1 311 1 34528021 6 1 1250200 
01302230412222812,1611151200 
121011311131119171101125100 
101011021210118131208100010 
1140434442027154432449 . 3700 
101123413221328141401114210 
103100010131119121152511010 
115112313132364160303220300 
112000320122355231402151100 
102110021131337130602111220 
101123332133119113141121140 
120011221130111121604112110 
014111121210711002306567314 
014111221122922111301331110 
113112311121246122402031400 
113111301141215030231116100 
113101221131127012700062200 
003122122123112230223212132 
113233222131102015541255440 
015011101122112001622110800 
113121130221109071111510112 
114100001222433112971251255 
125101420140109010901112500 
114012211131119010901231300 
113111231130019100906000130 
123101310231119021702112400 
01410330232.555111252350000 
014113302141228022301131310 

10320122112222821250 . 00 

102123321123343243100122320 
123112113230335134111132210 
102023212321426112330142210 
015100110124173113411051300 
014144404243209000001010070 
114133311324537123401331200 
102111422121119018102331100 
101121123130109009005050000 
002022302142491014410161110 
013111.00121302022. .1151110 
100013231113416222220220222 
001122131140009000905050000 
001100020122955224103220030 
000011120141101031510123310 
002133113221422114302121110 
002111231111109120702151100 
002011310131112010102112210 
123011111231127152205111200 
113222424202991513107010100 
101111121121337100275011210 
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141 

45770 

140 

45576 

139 

39270 

138 

46100 

137 

45571 

136 

46630 

135 

46270 

134 

45499 

133 

46170 

132 

45177 

131 

45274 

130 

46250 

129 

45299 

128 

, 

127 

45452 

126 

46270 

125 

45475 

124 

45274 

123 

122 

46150 

121 

45299 

120 

45255 

119 

•45254 

118 

45571 

117 

4024 

116 

45672 

115 

4024 

114 

45451 

113 

4016 

112 

4016 

111 

4016 

110 

4011 

109 

4016 

108 

45799 

107 

45750 

106 

45770 

105 

46170 

104 

45750 

103 

46250 

102 

45452 

101 

45770 

100 

4096 

099 

45470 

098 

41199 

097 

4024 

096 

, 

095 

45274 

094 

46650 

093 

45799 

092 

46200 

091 

46170 

090 

45200 

089 

45174 

114103403231119024301024300 
113022303221363166903486800 
114113313112428262201132210 
01501331312.34624130.3312. . 
114121412112682521200112510 
100000100221244432113352223 
112111111101299222221112131 
114112312140307024314111111 
110201410133119630102020060 
113122331131373024221330300 
014011412224564141221132120 
001023103211327111701131400 
015112302233208051311011430 
0021003201310072539.459920. 
0020111021112191109 . 255190 . 
103011212130328132603131200 
114211313321437121511220410 
003033304231781015400121510 
001000310131191100091001800 
002011222013901120133112012 
013000331121137052301261100 
001001210322730006200031400 
014111321112272124111111231 
113012331221337037000025210 
120011111122436221112332341 
114122332231373224100011107 
123011303230101031604120300 
114331114121119012601161100 
124011231141101000001012000 
124113312121119140501040320 
124223434121237215203113110 
123. 11331221 .30127000151030 
124011320131128005051052300 
114032314221523071200142210 
114112222114827692103114240 
114103414202391210601141300 
013012313141101000231131310 
1030112111202111122.2112210 
101022210122291115111071100 
003123211323419201431235190 
114011131130119070124110121 

124011222141 . 

103144404201773005500000550 
111113322224516015400441100 
121111131141191132401331200 
00301331411221100179122851 . 
015111331130118071205111110 
103211231131319152112230200 
114133404222328143201231210 
015111303131228011700111700 
0142111021203212312 .1221.3. 
015211313131219031601231300 
002111332134219104501081000 


80 


088 

4021 

087 

45599 

086 

45174 

085 

4016 

084 

4024 

083 

4024 

082 

4024 

081 

4024 

080 

4024 

079 

45177 

078 

4091 

077 

45473 

076 

45255 

075 

45872 

074 

45274 

073 

45650 

072 

45490 

071 

45470 

070 

45470 

069 

4024 

068 

45255 

067 

46290 

066 

45275 

065 

45570 

064 

4024 

063 

4024 

062 

4024 

061 

4024 

060 

4024 

059 

46230 

058 

45471 

057 

45254 

056 

4011 

055 

4024 

054 

46290 

053 

4096 

052 

4024 

051 

4021 

050 

45671 

049 

048 

4021 

047 

4091 

046 

-  45470 

045 

4024 

044 

45275 

043 

4096 

042 

4024 

041 

46350 

040 

4024 

039 

038 

, 

037 

122022331224173117011403200 
01311320130.682502300260200 
013012203122437122322121211 
123123332121346305201225000 
12111131114.811032501131400 
123200240131101090100910000 
122112121140119044202222110 
121012323130109090009110000 
125211221131118022151114101 
1131343042227010104.113323. 
12001142113183703340113.310 
113111102121221220901000900 
002144404312529000991569979 
013112101121246113501010270 
014122404124223125721664660 
102100110132055091000000000 
115122022232612057181121310 
102111311122326223121151110 
0022003001223212101.343111. 
123001131141218051400131410 
015111402134391001180021403 
014111311131219011801112500 
014100000121337022601221400 
013223033131228122300131410 
12501123113.2171333.522100. 
120001310141119034301221220 
121011422131125123103121111 
121114231141219162102016010 
121111321131137048222411110 
000011111021211017201111213 
013111441141109005050122104 
001000130301319031600071200 
1230222222302111432.133111. 
120011312131309002800611200 
014011111221155010900010900 
122111221131118031601022410 
121012322121255141401231210 
123001210121264152201123210 
014110211131128122502131210 
0021334121315211112 . 166445 . 
120014313132237043301060120 
120012301121219042401023400 
013013212031136222221151110 
121111322130319142305312111 

013212212121355221502131120 

125211311121219022601121310 

122012312231309001906010300 

101122221211855040600041110 

121011131131119033402121220 

001111102212594957833444570 

114111131140228024403131110 

101112231133382132311132120 


81 


036 

035 

034 

033 

032 

031 

030 

029 

028 

027 

026 

025 

023 

022 

021 

020 

019 

018 

017 

016 

015 

014 

013 

012 

Oil 

010 

009  ■ 

008 

007 

006 

005 

004 

003 

002 

001 

191 

4011 

192 

45471 

193 

4024 

194 

195 

4016 

196 

197 

45470 

198 

46130 

199 

4011 

200 

45491 

201 

45770 

202 

45799 

203 

45850 

204 

45710 

205 

206 

4016 

207 

45699 

208 

4016 

11410032010211900.9010123.30 
115211102221473113221331110 
013112321121190210701001080 
1021112210411090307011313 . 0 
113000030131118052302121210 
003000321321122602200053200 
015113403233355133300030250 
112012202131473022600010810 
011133223424973017200080101 
002022231130122011204111120 
0140222221301050113.415252. 
101111401212328139300644522 
102011211141109000001141200 
000011122134119221301141210 
114111111131364214301121500 
1130131111411010031 . 222242 . 
000012120040109000306188070 

1240 . 

123101210131119011801133110 
120011121141109000092123110 
013021322221215222201151100 
113023402221312133022151100 
114012113100203997659998504 
013111334041219000901031500 
011022331141119020801010800 
000011121122122101201221310 
003022202111364002351052200 
123111313234212022104333310 
125243424221664223301011214 
122134424212541141406111100 
12301133123151103331112121 . 
121101211130155050501132210 
002222411322773108101012410 
00101123112125522050.55554. 
014134414311619023501111610 
0230002201312210201.511110. 
113113321231527063100151300 
020011211131118113501131310 
023112211141109009001161100 
023100040141219050501331110 
002101130130219222405121100 
114113330122176510333213220 
000011112134212123302344210 
023215111322.55324105121100 
01302231212.591117101070110 
114011222131537135103122110 
1 141334122247730244 .011134. 
1022113111319131401 . 212311 . 
113000421221319030702341100 
102022433221514050410017111 
120012303321312154401223200 
114112301231309001810117100 
1252233122313120316031222. . 


82 


46200 


209 

210 
211 
212 

213 

214 

215 

216 

217 

218 

219 

220 
221 
222 

223 

224 

225 

226 

227 

228 

229 

230 

231 

232 

233 

234 

235 

236 


4024 

45770 


45899 

4024 

4011 

45274 

46270 

45750 

46690 

4011 

45299 

4016 

45571 
45850 
46250 
4016 
46290 
45274 
45491 

45572 
45453 

46152 

4021 


115101211131119123221212211 

11402212113011111111112111  . 

12110131113416303340100342. 

102022102131118142300134110 

1231334131221281324:21331: . 

114011122131191162000010900 

022133201231419133300114400 

022111211131219141313131110 

013110301122373521201133200 

002012102141337013400000550 

002022332120428331213213110 

1011114121424370325 .112240 . 

1140443142215190429.1110700 

023111221134482015400124300 

014122304222364122501011700 

123111211141119010881151110 

1132111301411110121.11111.  . 

002112301222437127000252100 

002021132041719134881565408 

120011121231219122231151200 

015111321131218113501331200 

01310151222121100090. .334. . 

013111221141109010451141120 

013122301234491010270090000 

000012212111313432542111131 

12101212123422101221122211 . 

101013113121433122321222210 

123033222131119112241111123 


237 

238 

239 

240 

241 

242 

243 

244 

245 

246 

247 

248 

249 

250 

251 

252 

253 

254 

255 

256 

257 

258 

259 

260 


4096  123013401124211200613341300 

45255  002022333134109033301090000 

45491  114033313131101000017110101 

46170  103013131140337120704112110 

46250  102011211122119212501222210 

013021320124361003160021007 
013132213234337011801011700 
023113303221618114401221400 
46290  014111404222281050501121410 

4016  024013322231437441102024110 

4096  024110221131119111704111300 

02001132114. .09040401111420 
45470  103121233131028120701120420 

4016  023111404121519123500005500 

4024  02301140312135513312112240. 

45750  101013313102228071201213300 

0142112211411010118.1241100 
46350  102112131141328090005111020 

45434  100013222120325321311231111 

102122221131219111291421200 
114143414121355112334121610 
46350  102112212121109142301142110 

45799  113123312130719011802111320 

42571  113113302011319310601110700 
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261 

04024 

262 

263 

4016 

264 

45274 

265 

46170 

266 

, 

267 

4024 

268 

45471 

269 

4016 

270 

, 

271 

4016 

272 

273 

4011 

274 

275 

276 

4024 

277 

278 

4024 

279 

46370 

280 

281 

4021 

282 

283 

4024 

284 

285 

4021 

286 

287 

4024 

288 

45770 

289 

4011 

290 

4024 

291 

4096 

292 

293 

45475 

294 

295 

296 

44670 

297 

298 

45770 

299 

300 

4096 

301 

302 

4016 

303 

304 

305 

45475 

306 

307 

308 

309 

310 

4021 

311 

4016 

022012311221202050503070000 

023001220131119011171011114 

02301221323162211 0441 1611 00 

0031113201221281333.0111115 

015011321211764621101341100 

002011400212782221301241200 

021022221121237161113112210 

014211231121234151203033100 

02510140113.319032501030330 

023013312221319141401113310 

023011311131137141403112210 

023011121121219171104212010 

022033122133219033311111231 

014201100131218121601050400 

022111322130129225205121110 

0200111211212010110.006350 . 

0222111211311090118.111122. 

025011322132828213404311100 

111113214133119061301230130 

013021212121373122111111240 

020003401234364114401230220 

024111321144309091005005000 

021113404143209052031143100 

022111331134426021162111113 

020003311141219062201122220 

10000111001135322242358542 . 

0211012201401190604.123210. 

112232103114420051400132310 

023211110141212131412221210 

02401230113121306099126190. 

125034413231109003601112300 

011111212041319011801352870 

1031124042116423339 . 2212244 


104111111131119191401463730 
020011101134119010901211600 
114022313124219117102131300 
1011210421213281215.2011006 
103122212241119023501125100 
1130223231311280028.014041 . 
123011221121119133301142110 
114012413234312000731030140 
125223414231237126101151110 
0140333042313161214.235572. 
024031312121318112421011610 
11313341321238123130142101 . 
11511140313151904150011251  . 
102012211122255121331231111 
121112231131237152113131110 
022012121131137153102150200 
02201232222154116120113221 . 
0230111112214151225.1131220 


84 


312 

313 

45730 

314 

4024 

315 

45770 

316 

, 

317 

45574 

318 

45572 

319 

320 

321 

4024 

322 

30594 

323 

324 

45470 

325 

326 

45740 

327 

45235 

328 

45156 

329 

4024 

330 

331 

45473 

332 

4024 

333 

334 

335 

4024 

336 

4024 

337 

4021 

338 

45275 

339 

4024 

340 

4024 

341 

45751 

342 

343 

4016 

344 

45770 

345 

346 

4024 

347 

348 

349 

4024 

350 

45651 

351 

4024 

352 

4024 

353 

4024 

354 

4016 

355 

4024 

356 

45770 

357 

358 

359 

360 

361 

4011 

362 

4024 

363 

45770 

364 

45571 

0121111.21.0125122041130110 

114223132131109043301123010 

421113412131219031601021420 

112012210141119000901112410 

023113402221327022602222200 

0122111410401020100 .1010114 

103122231121211015131051300 

110111211231219151302241100 

012110121131419110803111440 

022111431121355181002061100 

011012210122363217001440000 

0241012301411190011.4221100 

0021234241312110012.3121403 

101012111111327151303041200 

1030010001 . 100900081100522 . 

001111321141137151301041310 

00112143013122813242131212 . 

021013303231426123400033400 

10203340141124671009851763. 

113211301214673110356000220 

020011311131764115121171000 

0231001301211190111.1111.  .  . 

120012331131119022601432000 

021111131131228022601141210 

121112230134118042401131220 

023111321134273010901122300 

002022312130155206201151200 

021111311134609000001121410 

02310243414333502314102115. 

002232412131219123400123400 

001044444402  ..  1  ....  00 . 0 

021000121120101232302121220 

113111401221573214302131210 

11411323211 . 1370505 . 225001 . 

0252113031123724222.212032. 

014121112231217143291360009 

012211220122251215311241520 

022100300131128111701121500 

10210032013.228140.11040410 

021112312133209011800121510 

021111311131312061101222210 

021101410131255131502141200 

022101220131211031601151110 

. . . .22.141414191135.111133. 

1131113011 . 1228032500114400 

120013412131412114402112211 

0031111111214270524368661.  . 

023011221033101000301112220 

122113424121209130601033310 

023011411131228111161111150 

0221133111231180333.111133. 

112111332121119152201144000 

102114414244728080201211102 
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365 

366 


367 

368 

369 

370 

371 

372 

Note ; 


002133300101728260202311210 
45870  012022222 . 


10212222101100111710011  .... 

4024  021122321124191010180141200 

1131123221204221221.323211. 
45474  102011202121437133301134300 

0211133112214321323.103321 . 
000011222212343332113150100 

<.>  indicates  missing  data  point. 
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Appendix  C: 


Open-Ended  Question  Responses 


Question  28 

Response  Characterization  Number  of  Responses 

Make  more  user  friendly  66 

Eliminate/Reduce  multiple  screens  47 

Decrease  CAMS  downtime  46 

Provide/Improve  CAMS  training  24 

Decrease  CAMS  response  time  23 

Provide  user  defined  screen  capability  23 

Provide  on-line  help  20 

Re-evaluate  data  collection  practices  17 

Eliminate/Replace  CAMS  16 

Improve  AFM  66-279  15 

Set  up  remote  CAMS  flight  line  access  13 

No  changes  13 

Use  pull  down  menus  9 

Eliminate  automated  forms  or  781 ‘s  7 

Question  29 

Response  Characterization  Number  of  Responses 

Inadequate  Training  93 

Not  motivated  to  make  correct  entries  37 

Not  user  friendly  23 

Difficulty  in  using  multiple  screen  entry  22 

CAMS  won't  accept  correct  information  21 

Insufficient  time  for  correct  data  entry  19 

Difficulty  in  using  -06  WUC  manual  13 

Not  proficient  with  CAMS  11 

Software/Hardware  Malfunctions  10 

Not  enough  terminal  access  10 

CAMS  changed  without  informing  users  10 

Keystroke  errors  9 

Human  error  9 

Technicians  don't  perceive  benefit  8 

Inability  to  correct  wrong  entries  7 

Poor  CAMS  documentation  7 

Frustration  6 

Question  30 

Response  Characterization  Number  of  Responses 

Work  Unit  Codes  72 

Action  Taken  Codes  69 

How  Malfunctioned  Codes  62 

Time  Accounting  56 
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Question  30  (cont.) 


Serial  Number  27 

Part  Number  23 

Narrative/Corrective  Actions  20 

Type  Maintenance  12 

When  Discovered  Code  11 

Job  Control  Number  6 

Question  31 

Response  Characterization  Number  of  Responses 

CAMS  won't  accept  correct  entry  73 

Insufficient  time  for  correct  entry  56 

Don't  intentionally  input  incorrect  info.  54 

Just  don't  care/not  motivated  53 

Inadequate  Training  30 

Inflating  manhours  20 

Supervisor  pressure  to  alter  input  19 

Covering  time  for  8  hour  shift  12 

Avoiding  "difficult"  entries  8 

Delay  in  entering  due  to  CAMS  downtime  8 

Difficulty  in  using  multiple  screen  entry  7 

Question  32 

Response  Characterization  Number  of  Responses 

Provide  training  79 

Make  more  user  friendly  48 

Include  on-line  help  17 

Eliminate  multiple  screen  entries  15 

Improve  CAMS  documentation  12 

Simplify  system  12 

Eliminate/Replace  CAMS  11 

Train  special  cadre  for  CAMS  input  8 

Reduce  CAMS  downtime  8 

Provide  user  definition  capability  7 
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